{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "1 DL One Layer NN for DigitRecognizer.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [
        "f87cEn1_xfqI",
        "4RIGVSojmy-R",
        "4keUL7d0gBQZ",
        "X9uiFJ-ogBRi",
        "u13ffZdjrqQf"
      ]
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "metadata": {
        "id": "JS6jP7tpCrLr",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Digit Recognizer\n",
        "Learn computer vision fundamentals with the famous MNIST dat\n",
        "\n",
        "https://www.kaggle.com/c/digit-recognizer\n",
        "\n",
        "### Competition Description\n",
        "MNIST (\"Modified National Institute of Standards and Technology\") is the de facto “hello world” dataset of computer vision. Since its release in 1999, this classic dataset of handwritten images has served as the basis for benchmarking classification algorithms. As new machine learning techniques emerge, MNIST remains a reliable resource for researchers and learners alike.\n",
        "\n",
        "In this competition, your goal is to correctly identify digits from a dataset of tens of thousands of handwritten images. We’ve curated a set of tutorial-style kernels which cover everything from regression to neural networks. We encourage you to experiment with different algorithms to learn first-hand what works well and how techniques compare.\n",
        "\n",
        "### Practice Skills\n",
        "Computer vision fundamentals including simple neural networks\n",
        "\n",
        "Classification methods such as SVM and K-nearest neighbors\n",
        "\n",
        "#### Acknowledgements \n",
        "More details about the dataset, including algorithms that have been tried on it and their levels of success, can be found at http://yann.lecun.com/exdb/mnist/index.html. The dataset is made available under a Creative Commons Attribution-Share Alike 3.0 license."
      ]
    },
    {
      "metadata": {
        "id": "LgcAAVmXgBPm",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import math\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt, matplotlib.image as mpimg\n",
        "from sklearn.model_selection import train_test_split\n",
        "import tensorflow as tf\n",
        "%matplotlib inline"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "EITzxKZRgBPy",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras import models\n",
        "from tensorflow.keras import losses,optimizers,metrics\n",
        "from tensorflow.keras import layers"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "4PihLjAggBP1",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Data Preparation"
      ]
    },
    {
      "metadata": {
        "id": "cra90fdEsmsI",
        "colab_type": "code",
        "outputId": "9b0f8661-e512-40d2-dc4c-20d61f675e5f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        }
      },
      "cell_type": "code",
      "source": [
        "from google.colab import drive \n",
        "drive.mount('/content/gdrive')"
      ],
      "execution_count": 230,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Drive already mounted at /content/gdrive; to attempt to forcibly remount, call drive.mount(\"/content/gdrive\", force_remount=True).\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "K58DeQH2gBP2",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "labeled_images = pd.read_csv('gdrive/My Drive/dataML/train.csv')\n",
        "#labeled_images = pd.read_csv('train.csv')\n",
        "images = labeled_images.iloc[:,1:]\n",
        "labels = labeled_images.iloc[:,:1]\n",
        "train_images, test_images,train_labels, test_labels = train_test_split(images, labels, test_size=0.01)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "At9soAW0qOs4",
        "colab_type": "code",
        "outputId": "bf6f9b5b-32ed-4fe6-96a7-5e4e8a7f1bd6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "print(train_images.shape)\n",
        "print(train_labels.shape)\n",
        "print(test_images.shape)\n",
        "print(test_labels.shape)"
      ],
      "execution_count": 232,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(41580, 784)\n",
            "(41580, 1)\n",
            "(420, 784)\n",
            "(420, 1)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "f87cEn1_xfqI",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Keras"
      ]
    },
    {
      "metadata": {
        "id": "UfkBgDM1gBP5",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#### convert the data to the right type"
      ]
    },
    {
      "metadata": {
        "id": "-X3Uu-o_gBP6",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "x_train = train_images.values.reshape(train_images.shape[0],28,28,1)\n",
        "x_test = test_images.values.reshape(test_images.shape[0],28,28,1)\n",
        "y_train = train_labels.values\n",
        "y_test = test_labels.values"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "7vboLIlsgBP9",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "plt.imshow(x_train[12].squeeze())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "b6I1adl5gBQD",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#### convert the data to the right type"
      ]
    },
    {
      "metadata": {
        "id": "GAIvNCv6gBQE",
        "colab_type": "code",
        "outputId": "cd0ed9cc-9597-49e9-805c-7ca8276d9d75",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "cell_type": "code",
      "source": [
        "x_train = train_images\n",
        "x_test = test_images\n",
        "x_train /= 255\n",
        "x_test /= 255\n",
        "y_train = train_labels\n",
        "y_test = test_labels\n",
        "print('x_train shape:', x_train.shape)\n",
        "print(x_train.shape[0], 'train samples')\n",
        "print(x_test.shape[0], 'test samples')"
      ],
      "execution_count": 107,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x_train shape: (41580, 784)\n",
            "41580 train samples\n",
            "420 test samples\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "collapsed": true,
        "id": "xeMZR7ntgBQI",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### convert class vectors to binary class matrices - this is for use in the\n",
        "### categorical_crossentropy loss below"
      ]
    },
    {
      "metadata": {
        "id": "8e2qjHPLgBQJ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "y_train = keras.utils.to_categorical(y_train)\n",
        "y_test = keras.utils.to_categorical(y_test)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "LT78eGccgBQN",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Creating the Model\n"
      ]
    },
    {
      "metadata": {
        "id": "AMOStnPCFWSI",
        "colab_type": "code",
        "outputId": "4abba544-ab01-4992-be57-e56ac0d5cf82",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        }
      },
      "cell_type": "code",
      "source": [
        "model = models.Sequential()\n",
        "\n",
        "model.add(layers.Dense(units=10, activation='softmax',input_shape=(784,)))\n",
        "# model.add(layers.Dense(units=10, activation='softmax'))\n",
        "\n",
        "model.summary()          "
      ],
      "execution_count": 109,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense_21 (Dense)             (None, 10)                7850      \n",
            "=================================================================\n",
            "Total params: 7,850\n",
            "Trainable params: 7,850\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "BLdOV8vYgBQS",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "model.compile(loss=keras.losses.categorical_crossentropy, \n",
        "              optimizer=keras.optimizers.Adam(0.001), \n",
        "              metrics=['accuracy'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "pxI5Gi4xgBQV",
        "colab_type": "code",
        "outputId": "eaac424d-5b99-4307-e9f8-59b71f2aa0ca",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 3454
        }
      },
      "cell_type": "code",
      "source": [
        "H = model.fit(x_train, y_train,\n",
        "          batch_size=100,\n",
        "          epochs=100,\n",
        "          verbose=1,\n",
        "          validation_data=(x_test, y_test))"
      ],
      "execution_count": 111,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 41580 samples, validate on 420 samples\n",
            "Epoch 1/100\n",
            "41580/41580 [==============================] - 3s 80us/sample - loss: 0.7068 - acc: 0.8258 - val_loss: 0.4065 - val_acc: 0.9167\n",
            "Epoch 2/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.3731 - acc: 0.8997 - val_loss: 0.3354 - val_acc: 0.9143\n",
            "Epoch 3/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.3254 - acc: 0.9101 - val_loss: 0.3152 - val_acc: 0.9119\n",
            "Epoch 4/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.3023 - acc: 0.9159 - val_loss: 0.3009 - val_acc: 0.9143\n",
            "Epoch 5/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2886 - acc: 0.9200 - val_loss: 0.2944 - val_acc: 0.9167\n",
            "Epoch 6/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2786 - acc: 0.9230 - val_loss: 0.2904 - val_acc: 0.9238\n",
            "Epoch 7/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2718 - acc: 0.9251 - val_loss: 0.2915 - val_acc: 0.9214\n",
            "Epoch 8/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2655 - acc: 0.9259 - val_loss: 0.2856 - val_acc: 0.9119\n",
            "Epoch 9/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2616 - acc: 0.9265 - val_loss: 0.2863 - val_acc: 0.9190\n",
            "Epoch 10/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2573 - acc: 0.9272 - val_loss: 0.2851 - val_acc: 0.9143\n",
            "Epoch 11/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2539 - acc: 0.9282 - val_loss: 0.2832 - val_acc: 0.9143\n",
            "Epoch 12/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2512 - acc: 0.9290 - val_loss: 0.2813 - val_acc: 0.9190\n",
            "Epoch 13/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2486 - acc: 0.9303 - val_loss: 0.2882 - val_acc: 0.9190\n",
            "Epoch 14/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2463 - acc: 0.9310 - val_loss: 0.2830 - val_acc: 0.9190\n",
            "Epoch 15/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2442 - acc: 0.9307 - val_loss: 0.2841 - val_acc: 0.9143\n",
            "Epoch 16/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2426 - acc: 0.9322 - val_loss: 0.2840 - val_acc: 0.9167\n",
            "Epoch 17/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2404 - acc: 0.9326 - val_loss: 0.2887 - val_acc: 0.9214\n",
            "Epoch 18/100\n",
            "41580/41580 [==============================] - 3s 71us/sample - loss: 0.2387 - acc: 0.9335 - val_loss: 0.2840 - val_acc: 0.9214\n",
            "Epoch 19/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2374 - acc: 0.9336 - val_loss: 0.2804 - val_acc: 0.9167\n",
            "Epoch 20/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2364 - acc: 0.9332 - val_loss: 0.2828 - val_acc: 0.9190\n",
            "Epoch 21/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2351 - acc: 0.9337 - val_loss: 0.2811 - val_acc: 0.9190\n",
            "Epoch 22/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2336 - acc: 0.9342 - val_loss: 0.2789 - val_acc: 0.9190\n",
            "Epoch 23/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2327 - acc: 0.9350 - val_loss: 0.2795 - val_acc: 0.9238\n",
            "Epoch 24/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2314 - acc: 0.9352 - val_loss: 0.2845 - val_acc: 0.9190\n",
            "Epoch 25/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2310 - acc: 0.9347 - val_loss: 0.2836 - val_acc: 0.9119\n",
            "Epoch 26/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2296 - acc: 0.9361 - val_loss: 0.2770 - val_acc: 0.9214\n",
            "Epoch 27/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2288 - acc: 0.9354 - val_loss: 0.2823 - val_acc: 0.9214\n",
            "Epoch 28/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2278 - acc: 0.9357 - val_loss: 0.2819 - val_acc: 0.9167\n",
            "Epoch 29/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2271 - acc: 0.9367 - val_loss: 0.2834 - val_acc: 0.9119\n",
            "Epoch 30/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2270 - acc: 0.9353 - val_loss: 0.2821 - val_acc: 0.9214\n",
            "Epoch 31/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2255 - acc: 0.9366 - val_loss: 0.2836 - val_acc: 0.9167\n",
            "Epoch 32/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2248 - acc: 0.9368 - val_loss: 0.2824 - val_acc: 0.9167\n",
            "Epoch 33/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2241 - acc: 0.9365 - val_loss: 0.2868 - val_acc: 0.9119\n",
            "Epoch 34/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2235 - acc: 0.9368 - val_loss: 0.2847 - val_acc: 0.9119\n",
            "Epoch 35/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2232 - acc: 0.9370 - val_loss: 0.2809 - val_acc: 0.9190\n",
            "Epoch 36/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2228 - acc: 0.9374 - val_loss: 0.2825 - val_acc: 0.9119\n",
            "Epoch 37/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2221 - acc: 0.9370 - val_loss: 0.2825 - val_acc: 0.9167\n",
            "Epoch 38/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2212 - acc: 0.9376 - val_loss: 0.2802 - val_acc: 0.9214\n",
            "Epoch 39/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2209 - acc: 0.9373 - val_loss: 0.2912 - val_acc: 0.9143\n",
            "Epoch 40/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2199 - acc: 0.9382 - val_loss: 0.2846 - val_acc: 0.9190\n",
            "Epoch 41/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2196 - acc: 0.9384 - val_loss: 0.2870 - val_acc: 0.9143\n",
            "Epoch 42/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2192 - acc: 0.9379 - val_loss: 0.2806 - val_acc: 0.9214\n",
            "Epoch 43/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2187 - acc: 0.9391 - val_loss: 0.2854 - val_acc: 0.9119\n",
            "Epoch 44/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2183 - acc: 0.9388 - val_loss: 0.2879 - val_acc: 0.9095\n",
            "Epoch 45/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2176 - acc: 0.9380 - val_loss: 0.2850 - val_acc: 0.9143\n",
            "Epoch 46/100\n",
            "41580/41580 [==============================] - 3s 71us/sample - loss: 0.2168 - acc: 0.9394 - val_loss: 0.2861 - val_acc: 0.9167\n",
            "Epoch 47/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2166 - acc: 0.9391 - val_loss: 0.2890 - val_acc: 0.9071\n",
            "Epoch 48/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2163 - acc: 0.9384 - val_loss: 0.2872 - val_acc: 0.9167\n",
            "Epoch 49/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2161 - acc: 0.9396 - val_loss: 0.2864 - val_acc: 0.9119\n",
            "Epoch 50/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2154 - acc: 0.9394 - val_loss: 0.2903 - val_acc: 0.9190\n",
            "Epoch 51/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2154 - acc: 0.9397 - val_loss: 0.2847 - val_acc: 0.9143\n",
            "Epoch 52/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2147 - acc: 0.9400 - val_loss: 0.2870 - val_acc: 0.9167\n",
            "Epoch 53/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2142 - acc: 0.9394 - val_loss: 0.2858 - val_acc: 0.9143\n",
            "Epoch 54/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2139 - acc: 0.9402 - val_loss: 0.2866 - val_acc: 0.9119\n",
            "Epoch 55/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2134 - acc: 0.9405 - val_loss: 0.2863 - val_acc: 0.9167\n",
            "Epoch 56/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2132 - acc: 0.9398 - val_loss: 0.2913 - val_acc: 0.9143\n",
            "Epoch 57/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2130 - acc: 0.9404 - val_loss: 0.2887 - val_acc: 0.9190\n",
            "Epoch 58/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2126 - acc: 0.9401 - val_loss: 0.2884 - val_acc: 0.9143\n",
            "Epoch 59/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2122 - acc: 0.9402 - val_loss: 0.2932 - val_acc: 0.9095\n",
            "Epoch 60/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2118 - acc: 0.9398 - val_loss: 0.2870 - val_acc: 0.9190\n",
            "Epoch 61/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2117 - acc: 0.9404 - val_loss: 0.2911 - val_acc: 0.9143\n",
            "Epoch 62/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2111 - acc: 0.9406 - val_loss: 0.2909 - val_acc: 0.9143\n",
            "Epoch 63/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2107 - acc: 0.9405 - val_loss: 0.2941 - val_acc: 0.9095\n",
            "Epoch 64/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2106 - acc: 0.9406 - val_loss: 0.2949 - val_acc: 0.9143\n",
            "Epoch 65/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2106 - acc: 0.9404 - val_loss: 0.2898 - val_acc: 0.9095\n",
            "Epoch 66/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2104 - acc: 0.9406 - val_loss: 0.2918 - val_acc: 0.9143\n",
            "Epoch 67/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2098 - acc: 0.9410 - val_loss: 0.2921 - val_acc: 0.9119\n",
            "Epoch 68/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2093 - acc: 0.9408 - val_loss: 0.2880 - val_acc: 0.9095\n",
            "Epoch 69/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2094 - acc: 0.9409 - val_loss: 0.2878 - val_acc: 0.9095\n",
            "Epoch 70/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2091 - acc: 0.9411 - val_loss: 0.2863 - val_acc: 0.9143\n",
            "Epoch 71/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2089 - acc: 0.9407 - val_loss: 0.2921 - val_acc: 0.9095\n",
            "Epoch 72/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2082 - acc: 0.9418 - val_loss: 0.2885 - val_acc: 0.9143\n",
            "Epoch 73/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2082 - acc: 0.9413 - val_loss: 0.2960 - val_acc: 0.9071\n",
            "Epoch 74/100\n",
            "41580/41580 [==============================] - 3s 71us/sample - loss: 0.2079 - acc: 0.9411 - val_loss: 0.2927 - val_acc: 0.9071\n",
            "Epoch 75/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2076 - acc: 0.9413 - val_loss: 0.2918 - val_acc: 0.9071\n",
            "Epoch 76/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2076 - acc: 0.9416 - val_loss: 0.2896 - val_acc: 0.9119\n",
            "Epoch 77/100\n",
            "41580/41580 [==============================] - 3s 71us/sample - loss: 0.2071 - acc: 0.9420 - val_loss: 0.2946 - val_acc: 0.9167\n",
            "Epoch 78/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2068 - acc: 0.9416 - val_loss: 0.2946 - val_acc: 0.9143\n",
            "Epoch 79/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2069 - acc: 0.9417 - val_loss: 0.2914 - val_acc: 0.9071\n",
            "Epoch 80/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2063 - acc: 0.9414 - val_loss: 0.2945 - val_acc: 0.9119\n",
            "Epoch 81/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2065 - acc: 0.9422 - val_loss: 0.2945 - val_acc: 0.9143\n",
            "Epoch 82/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2060 - acc: 0.9415 - val_loss: 0.2927 - val_acc: 0.9071\n",
            "Epoch 83/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2057 - acc: 0.9420 - val_loss: 0.2927 - val_acc: 0.9119\n",
            "Epoch 84/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2055 - acc: 0.9421 - val_loss: 0.2992 - val_acc: 0.9048\n",
            "Epoch 85/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2057 - acc: 0.9421 - val_loss: 0.2919 - val_acc: 0.9167\n",
            "Epoch 86/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2054 - acc: 0.9415 - val_loss: 0.2943 - val_acc: 0.9095\n",
            "Epoch 87/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2048 - acc: 0.9424 - val_loss: 0.2959 - val_acc: 0.9048\n",
            "Epoch 88/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2047 - acc: 0.9430 - val_loss: 0.2943 - val_acc: 0.9071\n",
            "Epoch 89/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2044 - acc: 0.9419 - val_loss: 0.2975 - val_acc: 0.9048\n",
            "Epoch 90/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2044 - acc: 0.9425 - val_loss: 0.3014 - val_acc: 0.9071\n",
            "Epoch 91/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2043 - acc: 0.9418 - val_loss: 0.2973 - val_acc: 0.9048\n",
            "Epoch 92/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2043 - acc: 0.9421 - val_loss: 0.2982 - val_acc: 0.9119\n",
            "Epoch 93/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2039 - acc: 0.9428 - val_loss: 0.2977 - val_acc: 0.9095\n",
            "Epoch 94/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2039 - acc: 0.9426 - val_loss: 0.3007 - val_acc: 0.9048\n",
            "Epoch 95/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2036 - acc: 0.9424 - val_loss: 0.2996 - val_acc: 0.9071\n",
            "Epoch 96/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2033 - acc: 0.9424 - val_loss: 0.2975 - val_acc: 0.9048\n",
            "Epoch 97/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2030 - acc: 0.9429 - val_loss: 0.2969 - val_acc: 0.9071\n",
            "Epoch 98/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2029 - acc: 0.9425 - val_loss: 0.2995 - val_acc: 0.9000\n",
            "Epoch 99/100\n",
            "41580/41580 [==============================] - 3s 69us/sample - loss: 0.2026 - acc: 0.9429 - val_loss: 0.2997 - val_acc: 0.9119\n",
            "Epoch 100/100\n",
            "41580/41580 [==============================] - 3s 70us/sample - loss: 0.2026 - acc: 0.9427 - val_loss: 0.2989 - val_acc: 0.9071\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "CHDGYz_Mki4G",
        "colab_type": "code",
        "outputId": "1b6c61da-3632-4606-e1e0-3c829e1c58a9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "H.history.keys()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "dict_keys(['loss', 'acc', 'val_loss', 'val_acc'])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "metadata": {
        "id": "G1s_nCBWk-91",
        "colab_type": "code",
        "outputId": "b4917a58-50f3-4500-8851-9ba90db6cf34",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        }
      },
      "cell_type": "code",
      "source": [
        "plt.plot(H.history['acc'])\n",
        "plt.plot(H.history['val_acc'],'r')"
      ],
      "execution_count": 112,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7fd9762b8ba8>]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 112
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xl4lNXZ+PHv7JNkskLCvoOHTUQQ\nBQERte77rsWtttpXbW19a1/b2lZrrW2t2rr1Z13rgrjUpdYNtS5sKqCiIBz2PYRA1slk9uf3xzMz\nmUkmyRASMZP7c125SJ5tzpmE+zlzn+WxGIaBEEKI7GI90AUQQgjR+SS4CyFEFpLgLoQQWUiCuxBC\nZCEJ7kIIkYXsB7oAcZWV9R0etlNcnEt1ta8zi9Mt9MR698Q6Q8+st9Q5M6Wl+ZZ027Oi5W632w50\nEQ6Inljvnlhn6Jn1ljrvn6wI7kIIIVJJcBdCiCwkwV0IIbKQBHchhMhCEtyFECILSXAXQogsJMFd\nCCGy0LdmEpMQQnSVxkCYLzfsZcuueo4Y24chffP3+Ro79zTQGAxTmOukIM+J09H6mPRwJMrGnXXU\nNQQZ3Def0kI3FosFb2OI9dtr2V3tIz/PSbHHRZ+SXIrzXftTvbQkuAshvnUaA2F8/lCbxxiGQfK0\ndgtgsVhS9q/eUs17y7fz1cYqwpEoAG8v3cpxkwdx5sxhRKIGX6zbw+ot1QwozePw0WX0LspJXKPe\nF+TT1btZ+GU5WyrqU16/INdB31559O+VS0Gek0jUIGoYVFQ1snpLFY2BSOJYT46D/FwH5Xtbzj61\nWiz87srD6d87bx/eofZJcBdCtGtrRT1fb65mWL98RgwoxG5rO6MbCkfZXe1j/Y5a1m6rZfOuOgpy\nnQzpm8+QvvnkuuxEDYNoFPLcdkoK3RR7XKzfUctHK3ayXO8G4JARvTny4L6U5LtZuWkvqzZVUb7X\nRzAcIRCMEk162JDTYWXskBIOHtGLglwnb326hQ076gAY0DuPw0aX0bckl5cXbOSdZdtYvLKcxkAk\n5RovfrCBYf0KsNks7K7yUeczbzBWi4WJI3vTpySHuoYgtQ1BKmsaWbethrXbalrUv7TIzdSxfeld\n6GZLRT0bd9axt87P6MFFHDSoiAGlHry+INXeANGoeXxnyyi4K6XuAaYCBnC91npp0r4zgJuBADBP\na31/0r4cYCVwm9b6iU4stxBiHxiGQXV9gHAkSnG+C0fSNHfDMPA2hqiqC7C3zk80ajCsXwElBS4a\n/GFe/mgjH3yxg3gMdDqsjOhfSK9CN4V5TnLdduobQlR7A1TX+ams9VNTH0hpVbudNnbt9aHTBMJ0\n+vXKxemwsXxtJcvXVqbsKyvKoTCWFrHbmlrqNd4gX6zfwxfr9yS2HTqqN6ceOZRh/QoS2yYd1JvX\nl2zh3WXbGdYvn0kHlTJuWAmbd9Xz6eoKVm+pxoKF3oVuBvfNZ8yQYo4c15dCT8vUSTAUoaK6kYbG\nEFarBavVQkGek7Kk1v+B0m5wV0rNAkZpracppcYAjwHTYvuswP3AJGAv8KZS6hWt9fbY6TcDVV1S\nciG+heKBsqK6kRynjQGlnpT9jYEwVXV+ehfm4HK2v46IYRhU1vrZssfH2k172VvnJ8dlpyDPSX6O\ng2A4gtcXwusP4XLYKMhzUpDrxB+MUFXnZ0+dn52VDWyv9NLgDyeum+e243TY8AfD+AMR0q3aV5zv\nIhiK0OAP069XLsdPGcSOygZWb6lm9ZbqtOW1ACUFLtTgInoX5TCsbz6jBhXRv3cegWCEbbu9bK2o\nJxSOYrVaEnnovbX+2PviZuYh/Rk1sJDS0nyWryxnyapd+AJhxg4pZuywEgpyna2+X5U1jXy5YS+V\nNY0cOb4vg/u0zK077DbOnDmcM2cOT9k+uE8+Rx3Sn8ZAGIfd2u6nEwCnw8agMk+7xx0ImbTcjwVe\nAdBar1ZKFSulCrTWdUBvoEZrXQmglHoPOA54Qik1GhgLvN41RRfimxM1DHZXN7Jtt5cdlV6K811M\nHFVKYZ6TSDTKivV7+fCLnWzYUYsv0BREB/fxMHNCf0ryXXz8dQVfrN9DKGzmfovzXRR5XITCUYLh\nCBZgWL8CRg0qoqw4h1WbqvhMV7K7pnG/ym4BykpyGTO0BKfdSnV9gBpvgFA4Su/CHNxOG54cB70K\n3JQUuDEMgw0761i/o5aoAefPHslxhw1MCXaNgXAiPeHzh8nPdVCc76Igz9lqUMxx2TlokJmWyKjc\nFksijZOp0qIcjp08MOPj08lxZUe2OpNa9AWWJ/1cGdtWF/s+Xyk1CtgMzAY+iB13F3AdcFkmBSku\nzt2vFdFKS/e99zsb9MR6d6TOFVU+lnxVTllxDsP6F9KnJBerNXWl1C/XV/Lie+uw2az0KnRT5HGx\np7aRLbvq2VZRTyAYSTn+ybc1o4eUsLvax95aPwADSj0cXOahX+88yvc0sHR1Bc+8szZxzoBSD2OH\nlVBR5WPnnga27fbiclhxOe0EQxE+/rqCj7+uSBzvdtqYdnA/RgwopH9vD2UlOfgDEaq9AWq9AdxO\ns7WeH2utV9f5qfEGyHHZKSvOpbQ4h7698joUsAzDwDBo8T59k+Tvu+M6cotK/Ka11oZS6jLMVE0t\nsAmwKKUuBZZorTcppTK66P6s21xamk9lZX37B2aZnljv5nWuqvMz77/r2bCjltJCN2UluQwq83DE\n2D6Jj+/L1uzm8TfX0JjUos5x2Tl0VG+mjuvD4LJ8XvxgAwu/Kk/7mnabhb4leQwq8zCozMPA0jx2\n7mngs7WVrNlchctpY/akAcyeOICBzT6iXzh7BEtWVdDgDzFZlTKkT37KiI5khmFQvtfHuu01VFQ1\nctCgIsYOLcbpsKXWO9dB/+L0HXBDeue22Oata8Tb+lv6rSV/35mfk47FMNp+RoZS6hagXGv9UOzn\njcAhWusWJVBK3QGsAM4ChgMRYCBmZ+vVWut3W3ud/XlYR0/8I4Dsr3cgGGHhV+Vs2+3F5w/R4A9T\nXOhmcKmHUQMLWbuthlcWbCIQipCf68DbGEp0+tltFg5TZTjsVhZ8WY7TYeXMGcMxDINtu72s217D\n3roAYLZWDGBwmYfLThpN35JcqusD1DYEKfI4KSvOwWZNn2po8Iew26y42hjz3Bmy/XedjtQ543PS\nthYyabnPB24FHlJKTQJ2Jgd2pdSbmKmXBuA04C6t9byk/bcAm9sK7KJnMAyD2oYgnhxHSl62xhvg\nyw17sWDmTIsLXCxbs5u3P92Gt7HlWOfFSd97chxc/J1RzDi4H+GIwZ7aRlZurOKDL3Yk0hsDS/P4\n4RnjU8YRG4bB+h21LFlVwYYdtUwd14fjpwxKBPEclz2jccd5bkfH3gwhuli7wV1rvVgptVwptRiI\nAtcqpS4HarXWLwMPY94ADOAOrfWe1q8mskXUMKj3hQgEw5QUuFOCtc8fZleVj1qv2fqtqg+wZVc9\nm8rr8DaGcNqtDOtXwJC++Wwur2Pd9tq0ozVyXHZOnz6Uw8f0wZPjINdtx+p08MmKHazdXoPLYeOU\naUPIj6VfHHYL/Xrl0a9XHscdNhC9tYbtlV6OOqR/i9mEFouFUQOLGDUws849IbqbdtMy3xRJy+y7\nrqr3pvI6Xl24iQZ/iO+dPIZ+vcwWbDRq8MbHW/hoxU6q6wNEouavzGa1UFacQ5HHRUW1j6pYuqO5\n3oVuBpV5qKxpZEdlAwZmSmTUwEImqzJcTht7ahvZU+tnQO88Zh86kFx3avtDftc9h9Q543M6nJYR\nWWRvrZ95/11HOBxl2vi+TBzZG6fDhs8fZvOuOt5bvp3P1zV9+PrdP5dxxUmjOWhQEQ+/9jWrt1ST\n67IzpG8+xR4XToeN3TU+yvf4KN/ro8jjZNywEvr1yqUk35zkUuhxMrDMkzI+2ecPs213PX1KcilK\nMzlECLF/JLhnGZ8/zJ7aRmq8AWq9QcqKcxLTxVes38Mj//k6MZllxYa95Ljs5Oc4UsZSjxxQyFkz\nh1HfGOLxN9fw/15dhdtpwx+McOio3lxx8hg8Oam5ZsMwCIWjbS6mlCzXbUcNLu68igshUkhwzxLb\nd3t585OtfLq6IpEuiXM7bQztm8+arTXYbVYuPVExamARS1bu4uOvd9HgDzF2aDFD+uQzdlgJY4cU\nJ4brDSrz8ODLK6mo9nHxcaM4dvLAtEP5LBZLxoFdCNH1JOfeTRmGQWMUlnyxgy/W72HVJnOVh369\nchk9pJhijzlbcGtFPSs3VbG7upGy4hyuOXN82inZbQlHojQGwomOywOpJ/6uoWfWW+qc8TmSc+8O\n1m2v4fN1e/A2hvD5w0QiUQaWeRjer4DSohw27KxNrO1R72saJnjQwEJOnDqECSN6YU3Tsq7xBloM\nQcyU3Wb9VgR2IUTmJLh/S6zbXsOrCzfx9eaWCzKt2LC3xbYij5OjJw9kaJmH0YOLKCtuOTMx9Xjp\ntBSiJ5HgfoBU1flZrivZVF7HxvI6dlebHZrjhhZzwhGDKS3MSQwD3FrhjR3jY2jfAsYOLaZvSS5l\nZQU97mOrECIzEtwPgJWb9vLQq6sSo1ZyXXYmjuzNSVMHp51UM25YCeOGlXzTxRRCdGMS3LuQYRhs\nLK+joTFE31559C5wM3/pNl74YD02q4ULjhnJxJG9KSvOaXUxKSGE6AgJ7l2gqs7Pwq/KWbxyVyLd\nAuZMzkjUoMjj5NqzD2ZE/8IDWEohRDaT4N7JFq8s58m3NcFQFKfdytRxfehbksuuKnMWZ3G+i0tP\nVNLBKYToUhLcO0kwFOGZd9ay4Mty3E4bl5ygmDq2T9Y81UUI0b1I5NlPPn+YJat28e6ybVRUNzK4\nj4f/OXM8fdoZmiiEEF1JgnsH1XgD/HvRZhavLCcYimKzWjhu8kDOmz0i5cnyQghxIEhw30ehcJR3\nl23j34s3EwhG6F3oZtbE/syc0J+CPJnFKYT4dpDgnqEab4DFK3fxwec72FPrx5Pj4IITRnLUIf0P\n6AOEhRAiHQnubfA2hvhi3R6W6d2s3FhF1DBw2K0cN3kgZ8wcJo9YE0J8a0lwbyYaNfhsbSXvf74D\nvbWGaGzVzGH98pkxoT9HjCkjV4K6EOJbToJ7TDgS5cMvdjJ/6VYqa/wADO9fwKSDSpl0UCl9S2T0\nixCi+5Dgjvmw58feWM3Hqypw2K3Mmtif46cMSjw7VAghupueHdwjEazbt/Hipggfr6pgxIACfnTO\nhJRnfXY31u3biPbtB/ae/asVoqfb9yc3ZBH3k4/Ta8oEov/4B32Kc/hxNw/stq++pGTKBPL++PsD\nXRQhxAHWo4N78NXXALjq/Yf5xThLt3/aUM5Tj2OJRHA//QQEgwe6OEKIA6jHBvegz0/uso/xuvKw\nRyMM/ckPsNS0fApSt9HYiOulFwGwVlXhfPvNA1wgIcSB1GOD+7Jn3yIn2MjmmSfh++mN2LZuIf9H\nP4Ro9JsvjN+P/YvPsH+2DPtny7Bu2rjPl3C98RrWuloCJ54CgPvZp1o91lJZmXgt+2fL2r+p+f1Y\nKiv3uUz7wlJZCX5/5id4vd37ZixEF8uo100pdQ8wFTCA67XWS5P2nQHcDASAeVrr+2Pb/wzMjL3G\nHVrrlzq57B1WUe2j7o13AOh7zkn4zjwbx9JPcb39Js633iB48qnfaHk8/3cDOc8+nfjZsFqpfn8x\nkTFjM76Ge655fsMtt2HdvQvnf9/FWr6TaL/+qQdGoxTPPhLb7orEpvCYsVR/sARaeWCI51f/h+vl\nF6la8hlGnz77ULPMWCt2UTx1EoGzzsF7930ZnVP4/UuxrdVULfsKrD22jSJEq9r9X6GUmgWM0lpP\nA64E7k3aZwXuB04GjgJOU0oNVErNBsbHzjkR+GtXFL4jDMPgmXfWMn7Ll+aGo44Gmw3f//4cAMfS\nT77R8lhqa3C//CKR/gPwXXs9/nMvwBKN4n7mnxlfw7p1C84FHxCceiSR4SPxX3QJlmgU1/PPtjjW\ntnEDtt0VhMZPwHft9YQOnYR99dfYP2293o6FH2L11uN+8bkO1bE9rueexdrgxfXWGxCbNNYmrxfH\nRx9g274N2+Z9/5QjRE+QSZPnWOAVAK31aqBYKVUQ29cbqNFaV2qto8B7wHHAR8B5sWNqgDyl1Ldi\nqcTP1+1hzdoKxpWvITx6LEZpKQDh8QcDYP/qy2+0PK6X/4XF76fxe1fR8NvbqP/bg0R7l5qBNBDI\n6Bruec8A4L/4EgACZ52D4XbjnvtUi2Bp/2qFecz5F9Lw29touPlW8xqtpHEs9XXYY2ki97Mtr7ff\nDCPx2tY9ldj0mnZPcXy6BEvYfP7sN/37EqK7yCQt0xdYnvRzZWxbXez7fKXUKGAzMBv4QGsdARpi\nx18JvBHb1qri4lzs+7FUbmlpfrvHGIbBm08vZ3TFWpyhABx/XNN5pfkwfDjOVV9S2tuTmqIoL4c+\nfdr++F9RATt2pN83fDgUtXzwNQAvzAWbDc81P8ATL8vll8Ff/kLpxx/Auec2Hbt2LXi9TT9vg1KA\nF54Fj4eCK+aAx2PW5dxzsT/9NKV6Bcyc2XTOBjN4emZOM1/vzJNh6FByXn2JnIceNM9PtuaLxLf2\ntZrSjV/D1Kmtvw/plJdD377p0z6LFsGG9VBYCLW1lKz4FGYe3rS/ttYcs5/XNKGs6LOmTxkFG9aY\n9e0KFRXQuzfYOrFdEghAfb153dYYBuzaBf36pWzO5G8820idO64jM10S/0O11oZS6jLgMaAW2JS8\nP5aPvxI4vr2LVlf7OlAUU2lpPpWV9e0et2ZLNRu213KDz2yJ1k6aSjDpvIKxB+P6z6vsXbGG6ICB\nANhWraT42Bk0/PI3NP74hrTXtdTXUTJ5PNaamrT7wyNGUr1oWYubg+3rVZQsXUrg+BOps3sgVhbb\nGedT8pe/EPh//6Bu1gkA5N55B3l33tFq3RovvgRvowGN5jUcZ19I0dNP0/jwY3hHT0wcV/jJUpzA\nnoEjMGKvl3v+xeT9+Q/UPfYUgYvmpFw3Z8ESPID//ItwP/8sjQ8+hHfEuFbL0Zzj48UUnX4idff+\nncCF322x3/PgQ+QA9b+5jfz//TGBt96h7oLLzJ2hECXTJhEt60PN6++AxUJpaT6h+e9gt9uxhMME\nP1lKbQa/+31l3biBkpmH47vueny/+E2nXTf/Rz/E+fprVC9ZTrRP37THuF6YR8G1V1HzwquEZs0G\nMv8bzyZS58zPSSeTtMxOzJZ6XH+gPP6D1vpDrfVMrfWpmAF+M4BS6gTgV8BJWuvafSptF3n7060A\nTNq5CsNiIXTk9JT94YMnAKkf9V3z38QSjZLz+CMQSf/hw/XKS1hragjOmo3v6mtTvkKTJmPfsB7H\nogUtzounI/wXXZKyPaJGE5o8Bef772HduQPne/PJu/MOIoMGp1ybn/7U/P5HP8X3f79KuUZo2nSi\nhUU4P/ygaaNhYF/5JZHBQzEKmz5J+C+4GMNiIWduy9RM/L3wXXs9kQEDcb38L2hoaHFca5xvvm6+\nR2/8p+VOrxf3Ky8RGTQY/3cvJTJoMI7FCxIjlpzvvI1t6xYcyz5NpJOoqcH+5QrCk6cQGTTYLF9n\np4oA13vzsYRC5Dz5eKfNGbDs3YvrpReweutxPT+v1eOcCz4EIOexhzvldUXPlEnLfT5wK/CQUmoS\nsFNrnbi1KKXeBC7DTMOcBtyllCoE7gSO01pXdX6x91353gZWbNjL6DIXBV99RvjgQzCKilOOaQru\nKwieeDIAjoVmULbt2I7jow8IzT62xbXdc5/CsFio/+sDiRZ/nP3jJRSffgLuuU8RmjmraUcwiPuF\neUR79yb4nRNaXNN/8SXkL19K7t134nrtZQyXi7rHnyY8oakVnluaT0Nrd3mbjdC06bjeeh3rtq1E\nBw3GWr4T6969BKam3tSigwYTOuponB++j23DOiIjRjWV/6svMXJyiByk8F/4XfLu+hOu/7xK4IKL\n079uM/GbmmPJIvPmmJTicL32ChZfA/5rfwxWK6HpM3HPewbbqpVEDp6Q0g/gnvsU3gkT4aOPsESj\nBKfPxL76a1xv/gdrxS5zyYVOFP+9W/fuxTn/LYKnnr7f13S/9DyWUMj8/tmnaLzu+rSpqvgN1fnO\nW1h278YoK9vv1xY9T7std631YmC5Umox5kiZa5VSlyulzood8jDmDWAh5pDHPcAFmJ2tzyulPoh9\nDe6SGjQ2mjnMdsxfug2Ac117sASDhKbPbHFM+OBDgKSWeyCAY+nHRPPN/uN0nY42vQbH8qWEjj6m\nRWAHCB8xlfDwEbhe/zeW2qa0jfPtN7FWVeE/90JwtpwZGzjzbIycHHKefAxrdTXe2/+cEtgzEZph\n1jEeYOP1it/EksU7Y93PPpNUiAA2vZrw2HFgs+GPpVVynnwc26qV2FatxLp9W6uvb6mpTrS4rbU1\n2Fd9lbI/J3ZTjF83OOMoAJyLPsJasQvnu/MJjzuYSJ++uP71gjkO/v33zbrNnJVyM26XYWDZvTt9\nOffsSd0QieBYsrDV37ulrrbVT3EJPp95XNLru595CsNuJ3jUbOzr12Ff+mnL82LvOYAlHO6yEUrf\nNMvu3V07hyQSkXkPzWQ0QFhrfZPW+kit9Qyt9Qqt9RNa65dj+17SWk/UWh+qtX4mtu0fWuv+Wuuj\nk762dkUFCq68BEaPxlqxq9Vj6nxBFq/cRWmRmwmLzPRAPPAli/bpS7S0LBEsHJ8tw+L347/wYsKj\nDsL1xn+wVKd+EHHHxqfHg2MLFgv+iy/B4vebKQ2AYJDc++8xz2uW444z8gsInHamecx5F+K/5PJW\n69ea4PRYsIx9zI/XK11wD5x0KtHCIlzPzYX4SBS9Gks4THi8edOLDhlKcOYsHEs/oWT2kebX5PHY\nv/yixfUAHIsXYTEMwiPNTwLx1jCAbcM6HJ8sITTzaKKDzPt+KBbcHYsW4Hp+HpZIhMY5lxE4/yKs\ntTW43vwPvP8+hstFaPKUljfjNuT95pf0Hj+yxY3AsWgBvccOx5UURO2rvsJaU0Pg1NMJHToJ53vv\nYN1lZiJta1ZTcug4Cs86BWKt8HSKzjuD4iMPw1q+M1bGFdi/Xknw+JPwXXc9kL6xEH/P/aefheF0\nds0IpW+Yba2m16FjyL/2qi6rS+69d9PrkNFYt3VJmOmWuv3sj9DRx8DOneRf/b1EUGruw893EApH\nubpqKTkvPkfokEMJzjom7bHhgydg27EdS9VeHLGgGJoxyxw7HgzieumFpBcP4X7+WaLFxYmZoekE\nzr8Iw2pN/GfO+92vcXz+Gf5zzm9zolLDL36N9+ZbqL/zr61OMGpLZMxYor16mS13w0hquR/S8mC3\nm8A552Gr2IXz/XeB9C197+1/xnf1Nfi+fzX+s8/DYhi4n3wi7es7Fn0EgO9nN5k/L/yw6eVinxD8\nFzV1skb7DyA8bDiOxYtwz30Sw+UicPa5iRtgzt/vgxUrCE05AtzutH0k6bhefYnchx4wy7B4YWoZ\nF3xgXvvRh5q2xW5CoRlHpcwZsHjrKfjeHKz1dTg/Xkze79J3tNq/WoFj6SfYdldQcNUV5t9JrD/D\nf/EcQjNnERk4CNcrL6WOfkqqS2jmLAInnYpdr8H+2bI26/dt5376n1hCIdz/eh734490yWs4lizC\n0tiI87/vdsn1u6NuH9wbf/A/cNZZOBcvbHU1xE9X72bk3i1M+fvtRAuLqHvkn2lTIZCamnEsWmB2\nvE47Ev95F2LYbImZoADOd+dj3VOJ/5zzweVqtYzRvv0IHvsdHJ9/Ru6fbif3H38nfJAyg3YbogMG\nmiN0cjv4oBCrldCRM7Ht3IFt0wbsK78k2ru01VEaidTMM2YgStfSj4weQ8Ntf6ThD3dS/8A/iPTr\nj+vlF8HXcrSTc+ECDLebwCmnEx4+AsfHS8wbcDiM67m5RAuLCJx8Wso5oRmzsNbXYd+wnsDJp2IU\nlxAZOYrQ4VNxfPF57BizhR/t159or15tBnfb+nV4fnIdhsMRq1PqsfGfHcuXYVtjpkPiN6HQjKNS\n5gx4/vfH2Nevo/GyKwmPOojchx7A+dqrLV7TFfs0Fx51EI5PluD5zS9w/esFImV9CB7zHTPFdcHF\n5sSt/6Sen/yex298yX9z3U4wiPvFeURLSoj26oXn1zd1yc3KtmE90NSgEFkQ3LFY4PHHiQwdRu69\nd7dYMKuiykf1jt3c/MadWAN+6u9/iOiQoa1eLhQLZI6ln+BYvpTwhIkYRcUYffoQ/M4JOL5agfPt\nN7GtWU3OE2YrpPlol3Tix+Td9SeM3DzqHnu65ZjyLpDIY//nNWzbtpqBupVPAeGDDyE87mCc8980\n15/56ksMm43wmFaGPtps+C+8GGt9Ha7X/52yy7JnD/bVqwgdPg1cLjNoe+uxr/gc5/vvYqvYReDs\ncyEnJ+W85HRZ8vuanPaKp5uwWAiPn4Bt6+aU/oyEhgazpd3gpf5vD2Lk5rYa3CGWYguHcXy8hPCI\nkUT79ccoLCJwyunYN27A/fK/CE05Au8f/kzdo09h5OaSf/012Daub7qg34/7xeeIxIZvhkeMJOfR\nf2CtrSFw/kWJdfbj/QzuZiOUkt/z0Kxjmm6eK1ZgW7PafK00qQ1LTXXafgBLXW3Gk+EACAZT+wri\notEO5bSd77yNdc8e/OddRN3fH4VwmILvX4alam/bJzY0mP1pmWhsTPT9OBct7PZprM7S/YM7QGEh\ntY8+heF2k3/d1Vi3bE7s+nxtJde/fR+9Krfj+/ENBE84qc1Lhcebwd391BMtOl7jwabwkgsoOeoI\nnO+/R+jgQ4ikyWE3Fzz+RKK9egFQf899RA5S+1rLDom3cuNph7QpmTiLBf/Fc8yOvOfmYl+10iyn\n293qKf4LYkHq2dTWpWNxPLUxM+Vfx6IFiZZoun6KeOCODByUMroocPqZGLl5kJdH+NBJie2JT1or\nUztrAfLuuRP7mtU0XnkVgXMvIDxmHLa1axILlFkqK7HtKid49DFEe/XC/cKz2JctxeqtJxS/gSSV\nM9qrF3UPPwEOB5HRY6j/y9+weusp+N6liU8urrffMPP151+EUVRs3gRiN7Dk/pV4/4Xz48XYNqwz\nN0YiTe95Tk7TzdNbDxMnUnJQ8FbnAAAgAElEQVTUEZRMnYT7iUdT6mndvIleE8eQd9tvU98An4+S\naZMpOuPEjBdly7/+GkoOP6RFIM+95056jR+FdeOGjK4Tlxjue/ElhI4+Bt+Nv8C2fRt5d7T+zAFL\n1V5Kjp4Gs2a1ekwy28YNWGIB3Vq5G9tavU9lzFbZEdyByMET8P7xLqy1NRRceWnijzn/kQc5cv3H\nNE6dTsNNN7d7nejQYUQ9+dhiHWHJLcng8SfScNPNNF7xffPrez/A+5cMl81xOqn7+6PmZJ6zzm3/\n+E4SGTmKSFmfRH3SdaYm859zPobTSe4Df8Xia0jc7FoTHT6C4JEzcC78COvmTYntzoXmx+Ng7OYY\nPNL81/XvV3C+/QbhsePTjv4xysqou/8h6h58JGXYpOHJp+7hx2Hu3JSUWqsjZmJ57mhJCd7f/j5x\nrCUSwb7m65RzQpMOw3/uBVj37MFzq/k3kvx7D804Cu/v/kDtvJeI9h+Q2B449wIaL7sS+9cr8fzi\nZ0BTSzweyCNjx1H79PPU33M/kVEHpRQxfox73lwAbJs2tnjPG394Hb5rfgzXXEPj5VdiOBzkPPFo\nSuvU/exTWHw+3E//M6W16/rPq1grd+P4bDme3/yixXvdnGX3blyvvoS1qiqxfHT8vcx57GGzz+m9\n+e1eJ866qxznu/MJHTop0bfk++mNRIuLcf73nfQnRaPkX3sVti2bYenSREd2W+I3x/DoMUBq305P\nljXBHczWQeNFc3B8+QWeX/+CwAcfcdprD1GXX0LDw09k9ug5qzWxzoxhsxGaemTTPpsN3w0/x/un\nu82vP95F+NDJGZcvdPQxaWdpdimLJdF6h/aDu1HSi8CJp2Dduzej4yE5SDUNo3Qs/IhonofwRLOV\nbZSVER49BseXX5ijQS6e02p6KHD+RYSnTmuxPfidE+H01PHmrY2YSfSHnHtB4pNH82PtK5s6mOOf\nyhzLzQVP4zcjACwWGn94HeFDDm1RJu/v/0ho4qHkPPs0uXfegeOD/xKackRKIA/NnIX/u5e2rOcp\npxMtKDRHKEUiafs4jOISGm75PTzwAN4/30Pw+JOwr16FfYXZ/0Akkrg5WOtqcb3xWuLc+Kep8LDh\n5DzxaMqIoHTcLz6XWLMn+ZOY87/vYq00h5Emj3hqj+v5eVii0dS0pc1m9gNt25ryCTsu969/wfXe\nO0Q95qzLdJP/mrOvN4N742VXmuXdhzJms6wK7gDeO/5CeOx4cv75KH0uvxCAT3519z4tVRv/zxWe\nOAnD0/3Xtkh0QHryiQwd3u7x/oub0gdtpnFiAqeeQdSTj/u5udjWrcW+9BPs69cRmjoNYh2ZQCLF\nZTgc+M+5YF+rkVZk+AiM3LxEoI5LpAMuTK5Lais/eTRQZOw4QhPN4B0ePSbziUMuF3WPPEm0qIi8\nO+/AYhitD4ttLieHwFnnYttVjvOD99oezRQT/93EPyE4PnwfW/lOgjOPjm03g7J100acixYQnD6T\numdfJOrJJ/9n17e+MFtsATfD6SQ49UgcKz7HFkt1xV8rmufBsWRh+2P8k6/ndhM465yUXcHYpyJn\ns8Dt+PB9cv90O5EBA6l79ElzW/PgHgy2mCFtW7fW3HXsd4gMHJQyy3l/WerrMqvvvohGsXi7flmF\nrAvu5OZS99iTRPMLcPi8PDljDv3PbjvP3lz8P1dyi7c7i3eqhscfnNHa5/GOvMQ57cnLI3DWOdh2\nbKdk+mEUn/Id8zrTU9+/4Awzhxo88RSMWP/Dfot90rKt1dhiLThLRQXOd94mdMihRJLKHx49FsNm\na2q5f7WCaFFRYpx9vIWZboJbW6KDh1B/v9mnYeTmEjjjrHbOaNIUrJ9uarm38Z4HZx9nTup66UVo\nbEy0sBt++WuCU4/EueADrFu34H4uPtR0DpHhI6n/2wNYfD4KvjenxfBLwHxwi15D4KRTafzhdWaZ\n5j2NZfdunO+8RWj8BAKnn4m1puVktHTsn35ijng65fSUpS6g6e8iPtTYrFiQ/OuuBruduoefIHTU\n0VBUlJijEZd/7VWUTJuU0kls27AOw+UyZ1pPn4m1uhrbqpXtlrE9Fm89JZPHU3jxuZ0a4HMefYhe\nY0dktALq/si+4A5Eho9k9zMv8o9jr2LpiXMoK8pp/6QkgTPPoeHnv8QX+yPv7qJDh1F/933mx/tM\n2GzU//0R6u77fy3+Y7bGd8PP8X3/ahovuYLGS67Ad/W1KZ8AINZn8Ytf4/3tbftahbZf+6r/wRKJ\nmIGroQH3C+YEqBYTxNxuIgeNxr56FZbaGuwbN5j57Vh6yH/RHBpuuhnfj366z2UIHn8SdX9/hLoH\nHt6nT3vhiZMIjxmL863XsX/+WYt1f1qw2wlccDHWulrcz/wT15v/IaxGE550WNNQ1rlP4p43l6gn\nn8CpZ5jlO+1MfFdfg33dWvJ/9uMWI0oSndwXzSH4nRMSy07nzH0ykUZLTDLLIO3RtG5Sy0l6ETWa\naO/SxPwLiK0jVLGLxsuvJHzY4WZ/y6xZ2LZsToyEse7cgeu1V7DtKscRH05pGNjWrycybDjYbCmz\nnPeXY8kirDU1ON9/j9w2Fu3b5+suXoTF78f91OOdds10sjK4A3xWNJzXDjmZQ0d3YF0Otxvfz27q\nvNblt4B/zmWEJx2W8fGhI2dkvH4MmGPyG/5wJ967/ob3rr/RcNsdGCXN3j+Hw+xQGzwk4+tmInj6\nWTReeRX2NavJ//lPzXRAbAJUc+GDJ2Dx+XDFxqendBi73fhu+HlKp+m+CJxzPsFTTmv/wGQWC/6L\n5mAJhbDW1WbYx2H223hu/TWWYND8xGGxmOmxPA+5D9yLbecOs+M+aY5Ew69/R+iww3G/9GLqiBuf\nD1fsgTGhWbPB4cB/3oVYq6rIvetPGE4ngbPPS5pB3E7gjC8IN3hI+k+/FgvBGTOx7SpPDCNN3Ay+\ne1nTcbPNFTEdsc559/PPYomlW+LbrLsrsHrriYw0+ziSZznvr/hNzMjNI+/uP+Pch87ktsQ7gPfl\nmQ0dkbXB/Yt15nohh44qPcAlEd8E7y23E5o0GfcL87CvW0vglNNaLAwHTXn3eB45k2Da1fznXogR\n6+zPpDyREaMITj0SSyCAYbebncYAHg+BM8/GEgsYzT854XRS98g/E5OJHO+/h3XTRnKe+SdWbz3+\nCy9OjFCKt7gtgQCBk07FKOlFtF9/wiNG4liyuNXZ4JC0INyF3201DdiUmvmoaVTNxEOJjE2aUxEL\n7s6FH5k5/LlmDt+wWBLBPZ6Kiy9xER0wMDHLua0yZsKxaAGG00nNcy9jOJ3kX/ODNtdSykgkgi32\n8BtrVRXO+V33IPusDe5bK+pxO20M7tP1E4XEt4DLRd3D/yRabAb01iaWxftTHMs+Tfn5QDJ69yZ4\ngrkKaXhCZuWJp2CC3zkxpfM3Xu94qqa5aP8B5mSiUIiiC86i1xET8fzq/8xzkzqfI6PHEJp8WOya\nTdtD049KTEZrTWJBuDY++SXPe0g7qgZg/HiiJSU4Fi3AsWQRts2bCJx2JuGDDzFHNfl8ic7UyIiR\nSdc+ypzl/NlyOspSXYX9qxWEDjuc8BFT8d7+Z6zV1eT/7PoOXxPAum0rlkCAUGzkVfNJbJ2pIw/r\n+NYLR6LsqvIxtF8+lg6sySK6p+igwdTOfRHHwo9Sl1dOktxZabjdREaOSnvcN817y+8Jjx1H8OiW\nS0qnEzj7PBq2bTWXvkgSnnK4OTzz0MmtDjUNHX0MdY8+hfPdt5vOmzCR6NBhqWX60904PvrQXL8p\nfu6MmeQ8+RiORQsIT57S4trxBeGCs2YnOqrTiQwfSaRff5yLPsK+8ktzVE3zNJrVSmj6Ubhee4W8\nO8x+Gv/Fl+B8520cX35hrt8TS3Ek/x4Dp5xGzlNP4H5hHt7Dj2i1DG1xLFmMxTASnev+S6/A/fyz\n5qedrVs6nFq0x8obPOkUsNsSz2zoaCqwLVnZci/f6yMSNRhYKq32niY8eQqN1/9vq+kAo6CQSCyI\nhceOy2zuwzcgOmQovht/kXl5nE58N/6C6PARqdstFhqvuobwlLaDWvDU0/H+9YHEl/97P2hxTHjC\nRHPN+aT3Mj7+v/kolrjEgnDtDQe1WMyRLXv2tDqqBpomwTk+WUJkyFBC06antPrjaZnk4J5YtuGl\nF9KueZSJeL9Cos/AYqFxzmXmQnnPze3QNaFp2GZ45Cj8F1+KJRrFneZB9p0hK4P79kpzqJcEd5FO\nPBUTX8pYZC4xGe3Tj7Fu2oh129amry2bmxaEO+nUdq+V3Nna2s0g5ZiL5pit+alHYthsOBd+hH39\nOqK9S1NvDMlrHiVN6jIv0soyDM22Oxd+hJGTQygptRU4zVwCwz3vmdRx9JFI+uWfo9EW6+PY1psd\nyJERoxLPbEj3IPvOkOXBPa+dI0VPlJik9i3oTO2OgjOOwtLYSK8jJtJr8vimrykTzAXhzjmvzfWI\nkq8DmKNqWplbEBl1EJGyPik5fMOTT3jiJOyfL8e6bSvhZss6QPo1j9xzn6L3qEEt1uZxLPiQ3qOH\nkneLufSEpbIS++qvCU2Zmrraq8eD/8yzsW3bmujQxeul6ITZFM+amvqsh0iEgksvpNfkcSmTrmwb\n1mFYLObku9gzG2ybN3XKuPzmvh2fSTvZjkrzzRwgLXeRRuOlV2BpaGiRrxaZabzqGix+P5Y0z5Y1\nXG58rTxIvrno4CHU3/EXIqPHtD65zmLBe/e9WKqqUp50FppxVGKpiHT9JtHhIwhOm45zwYdYt2zG\nWluD5/9uwBII4PnVzwkfPIHw5ClYy3dScPX3sPh85D54L+GJhyb6KtI90Md/0SXkzH0q8djM/Bt/\ngiP2sJr8666m7qnnwGol9547cc1/CzBTSqFjjgPM0T3RQYMTq6E23HQz0b79zHH6nSwrg/v2Si9F\nHieeHEf7B4sexyjpRcOvftv+gSKt6NBheO++r1Ou5b/yqnaPCR7fcoZ5cPpMcv92F0DKM39Trn3R\nHJxLFpHz0AO45r+NJRCg4Sc/I/dvd1Hw/cuofut9Cq66AuueSnw/vA73U0/g+emPCE853HyNNGP0\nw4cfQXjESFyv/5vwvXfj/tfzhCZPwfB4cL3zNjn33UP4kEPJvfMOjJwc8wEiixYQOuY4LPV12Cp2\nEUx6DnN04CAabr4lk7dqn2VdWsbnD1FVF5B8uxBZLHT41MQDWCIjR6Y9JnDameakrkcewrZ1Mw03\n3Ijvl7/B93+/MpfKOHoqjk+W4D/jbBpuvR3vPfdhbfDi/OC/5qJ3aRaKMyedXWJ+Arj9VqIlJdQ9\n8k/q/v4okf4DyLvjNgp+cDnY7dTOfRHDbk90zjYfk9/Vsi64b4+lZCS4C5HFcnMJxYZihke2zLkD\niTWPAIIzZ+G78ZcA+H7yMwLHfgfr3r2ER47Ce8995gzfM8/B9/2rAVosepcscMFFGDYbhsVC3YOP\nEB0wEKN3b3Otf6sVa20N3tv+SGj6TMKTDsP+xedY6mqbRva08kmjs2VdWibemTpAOlOFyGoNv72N\n4MdLiLaRr/bd8HOMPA++6/+36fkAViv1Dz5M+O/347/wuylrATXccjtGQUFiUlk60T598f7lbxgO\nRyKXDhCecgR1TzyDbctm/Fd8HzBXwHR8+jGOpIeyNF/Xv6tkYXCXlrsQPUF48pS0E6mSRQcOouG2\nlot+GcUl+H6Z5gHnTie+m37d7munW58fWvYPhKYfBXffiWPhAmw7tgPpO4C7QhYGdy9Wi4X+vTv4\nUGkhhOgkocMOx3C5cCz8CEskQjTPQ7Rvv2/ktbMquBuGwY7KBvqU5OCw29o/QQghulJODqHDDsex\neCE4HIRHj211WYjOllUdqlV1ARoDYRnfLoT41ghNn4nFMLAEg62O7OkKGbXclVL3AFMBA7hea700\nad8ZwM1AAJintb6/vXO6isxMFUJ82wRnzCLvz38ASKw7/01ot+WulJoFjNJaTwOuBO5N2mcF7gdO\nBo4CTlNKDWzrnK4ka8oIIb5twpMmY8QemvJNrkKaSVrmWOAVAK31aqBYKVUQ29cbqNFaV2qto8B7\nwHHtnNNldiRGykjLXQjxLeF0Eoqt0hn+hsa4Q2Zpmb5A8qr3lbFtdbHv85VSo4DNwGzgg3bOSau4\nOBf7fnSClpbmU+sLYbXAmJFlWK09Yx330tLMn9eZLXpinaFn1jtr6vznP8Lrr1Ny9LR2H1LfWXXu\nyGiZRNTUWhtKqcuAx4BaYFPy/nTntKa6umPrLoP5ZlRW1tPoD2G3Wdm7t+XT3bNRvN49SU+sM/TM\nemdVnYeNgevGwN6GNg/rSJ1buxlkEtx3Yra64/oD5fEftNYfAjMBlFJ3YLbg3W2d01XCEQObLasG\nAAkhRIdkEgnnA+cCKKUmATu11olbi1LqTaVUmVIqDzgNeLe9c7pKJBrFbusZ6RghhGhLuy13rfVi\npdRypdRiIApcq5S6HKjVWr8MPIwZzA3gDq31HmBP83O6rAZJQuEodmm5CyFEZjl3rfVNzTatSNr3\nEvBSBud0uUjUkJa7EEKQZTNUwxFpuQshBEhwF0KIrJRVkTAckbSMEEJA1gX3qAyFFEIIsii4R6JR\nDAMcEtyFECJ7gns4YgBgk7SMEEJkT3CPRKIA2NtZt0EIIXqCrImE8Za73Z41VRJCiA7LmkgYjrfc\nJS0jhBBZGNwlLSOEENkU3GNpGWm5CyFENgX3eFoma6okhBAdljWRsKnlnjVVEkKIDsuaSBhvucs4\ndyGEyMLgLjNUhRAiq4K7zFAVQoi4rAnuEelQFUKIhKyJhCEJ7kIIkZA1kTAi49yFECIha4K7jHMX\nQogmWRMJJbgLIUSTrImEsvyAEEI0yZ7gHo1PYsqaKgkhRIdlTSSMt9xlEpMQQoA9k4OUUvcAUwED\nuF5rvTRp37XAHCACLNNa/0Qp1R94DHABNuCnWuvlnV34ZOGwrOcuhBBx7TZzlVKzgFFa62nAlcC9\nSfsKgBuBmVrrGcBYpdRU4AbgZa31bOAm4PauKHwyScsIIUSTTCLhscArAFrr1UBxLKgDBGNfHqWU\nHcgFqoA9QK/YMcWxn7tURNIyQgiRkEkk7AtUJv1cGduG1toP3ApsBLYAn2it1wL3ABcopdYADwO/\n6cxCpxOSVSGFECIho5x7M4noGWvB/xI4CKgD/quUOgQ4DXhea327UupU4C/A2W1dtLg4F7vd1oHi\nmBwOsyplpfmUlno6fJ3uprQ0/0AX4RvXE+sMPbPeUueOyyS47yTWUo/pD5THvh8DbNRa7wFQSi0A\nJgPTgZtjx7wDPNjei1RX+zIsckulpfl4GwIA1NX4cGJ0+FrdSWlpPpWV9Qe6GN+onlhn6Jn1ljpn\nfk46maRl5gPnAiilJgE7tdbxV98MjFFK5cR+PgxYB6wHjohtmxLb1qUSM1TtknMXQoh2W+5a68VK\nqeVKqcVAFLhWKXU5UKu1flkpdSfwvlIqDCzWWi9QSq0HHlVKnR+7zI+7qgJx8pg9IYRoklHOXWt9\nU7NNK5L2PQQ81Oz4cuDk/S7dPkg8Zs8qHapCCJE1zdzEY/YkLSOEENkU3GOP2ZOWuxBCZE9wj0Si\n2KwWLBYJ7kIIkTXBPRSJykgZIYSIyZpoGIkY2CUlI4QQQBYF93AkKsMghRAiJmuiYThiSHAXQoiY\nrImGZstd0jJCCAFZF9yzpjpCCLFfsiYahqOSlhFCiLisiYbhsKRlhBAiLiuCu2EYRKKGPGJPCCFi\nsiIahhOP2JOWuxBCQJYE91A4AsjDsYUQIi4roqGs5S6EEKmyIhomnsIkaRkhhACyJLiHwvHgnhXV\nEUKI/ZYV0VBa7kIIkSo7gnus5S4dqkIIYcqKaBhPyzgkuAshBJAlwT3xcGxJywghBJAlwT0Uz7lb\ns6I6Qgix37IiGsZz7vKYPSGEMGVFNAzJaBkhhEiRFcE90XKXtIwQQgBgz+QgpdQ9wFTAAK7XWi9N\n2nctMAeIAMu01j+Jbf9ZbHsIuCb5nM6WGOcuaRkhhAAyaLkrpWYBo7TW04ArgXuT9hUANwIztdYz\ngLFKqalKqXHAhcBhwNXAqV1R+LjEDFWrpGWEEAIya7kfC7wCoLVerZQqVkoVaK3rgGDsy6OU8gK5\nQBVwFvC81joMfBb76jJNM1Sl5S6EEJBZcO8LLE/6uTK2rU5r7VdK3QpsBBqBeVrrtUqpoUBEKfUW\n4ABu0FqvaOtFiotzsdttHakDYV0JQElJLqWl+R26RnfV0+oLPbPO0DPrLXXuuIxy7s0kch+xtMwv\ngYOAOuC/SqlDYsfYgJOA6cAjwJS2Llpd7etAUUzx0TI+b4DKyvoOX6e7KS3N71H1hZ5ZZ+iZ9ZY6\nZ35OOpkE952YLfW4/kB57PsxwEat9R4ApdQCYDJQAazRWhvAwlhLvsuEZG0ZIYRIkUk0nA+cC6CU\nmgTs1FrHby2bgTFKqZzYz4cB64A3gRNi54wGtnVimVuQx+wJIUSqdlvuWuvFSqnlSqnFQBS4Vil1\nOVCrtX5ZKXUn8L5SKgws1lovAFBKnaSUWhK7zLVdVH5AHrMnhBDNZZRz11rf1GzTiqR9DwEPpTnn\nt8Bv96t0GZLH7AkhRKqsiIbysA4hhEiVFcFdHrMnhBCpsiIaJtaWkZa7EEIA2RLcZYaqEEKkyIpo\nGJLgLoQQKbIiGkpaRgghUmVFcA9FZIaqEEIky4poGG+5OyS4CyEEkCXBPRSOYrGAVdZzF0IIIEuC\nezgSlc5UIYRIkhURUYK7EEKkyoqIGApHZaSMEEIkyYrgLi13IYRIlRURMSwtdyGESJEVwT0kLXch\nhEiRFRExHI5is2ZFVYQQolNkRUQMRQwcdknLCCFEXFYE93A4IksPCCFEkm4fEaNRg6gBdpmdKoQQ\nCd0+uMta7kII0VK3j4jycGwhhGip20dEeTi2EEK0lEXBvdtXRQghOk23j4jhqKRlhBCiuW4fEeUR\ne0II0ZI9k4OUUvcAUwEDuF5rvTRp37XAHCACLNNa/yRpXx9gDXCW1vqDTix3QlgesSeEEC20GxGV\nUrOAUVrracCVwL1J+wqAG4GZWusZwFil1NSk0+8ENnZukVNFYmkZecSeEEI0ySQiHgu8AqC1Xg0U\nx4I6QDD25VFK2YFcoApAKXUMUA981dmFThYKx1vukpYRQoi4TNIyfYHlST9XxrbVaa39SqlbMVvn\njcA8rfVapZQT+C1wBvDXTApSXJyL3W7bp8ID7KzxA1CY76a0NH+fz+/upM49R0+st9S54zLKuTeT\naCLHWvC/BA4C6oD/KqUOwQzqD2uta5RSGV20utrXgaLA3qoGAAKBEJWV9R26RndVWpovde4hemK9\npc6Zn5NOJmmZnZgt9bj+QHns+zHARq31Hq11EFgATAZOAK5TSn0MnAI8qJQat08lzlDTaBnJuQsh\nRFwmEXE+cC6AUmoSsFNrHb+1bAbGKKVyYj8fBqzTWk/XWk/VWk8FXgeu0Vqv6tyim2ScuxBCtNRu\nWkZrvVgptVwptRiIAtcqpS4HarXWLyul7gTeV0qFgcVa6wVdW+RUsvyAEEK0lFHOXWt9U7NNK5L2\nPQQ81Ma5l3eoZBmStIwQQrTU7SNiPC0jQyGFEKJJ9w/usbSMTGISQogm3T4iRiLxlnu3r4oQQnSa\nbh8RQ9KhKoQQLXT74B6JB3drt6+KEEJ0mm4fEROP2bN3+6oIIUSn6fYRUca5CyFES9kT3CUtI4QQ\nCd0+IkpaRgghWur2EbGp5S5pGSGEiMua4C7j3IUQokm3j4jxSUwOScsIIURCt4+I8UlMNknLCCFE\nQrcP7olJTJKWEUKIhG4fEROjZWScuxBCJGRBcI9it1mwWCS4CyFEXBYEd0NSMkII0Uy3j4rhaFSC\nuxBCNJPRY/a+zWZO6A+y9IAQQqTo9sH9+CmDKC3Np7Ky/kAXRQghvjWkySuEEFlIgrsQQmQhCe5C\nCJGFJLgLIUQWkuAuhBBZKKPRMkqpe4CpgAFcr7VemrTvWmAOEAGWaa1/opSyA48CI2Kv8TOt9cLO\nLrwQQoj02m25K6VmAaO01tOAK4F7k/YVADcCM7XWM4CxSqmpwCVAQ2zblcDdXVF4IYQQ6WWSljkW\neAVAa70aKI4FdYBg7MsTa63nAlXA08ANsWMqgV6dWWghhBBtyyQt0xdYnvRzZWxbndbar5S6FdgI\nNALztNZrY8eFYv/+BJjb3ouUlubv18pfpaX5+3N6t9UT690T6ww9s95S547rSIdqIgjHWvC/BA4C\nhgFHKKUOSdp/LTAJ+N1+llMIIcQ+yCS478Rsqcf1B8pj348BNmqt92itg8ACYDKAUupK4DTgTK11\nCCGEEN+YTIL7fOBcAKXUJGCn1jq+kMtmYIxSKif282HAOqXUcOCHwNlaa3/nFlkIIUR7LIZhtHuQ\nUuqPwFFAFLgWOBSo1Vq/rJS6GrgCCAOLtdY/V0r9AbgQ2Jp0meNjrXshhBBdLKPgLoQQonuRGapC\nCJGFJLgLIUQW6vYP62hraYRsopT6MzAT83d2B7AUeAqwYY5eukRrHThwJew6sQ77lcBtwHtkeb2V\nUt8Ffo7Zj/Ub4Euyv84e4EmgGHABtwK7gL9j/t/+Umv9PweuhJ1LKTUeeBW4R2t9v1JqEGl+x7G/\nhZ9g9nf+Q2v9aKav0a1b7m0tjZBNlFKzgfGxep4I/BVz7sADWuuZwHrgewewiF3tZsyZz5Dl9VZK\n9QJ+C8wATgXOIMvrHHM5oLXWszFH5/0N8+/8eq31dKBQKXXSASxfp1FK5QH3YTZU4lr8jmPH/QY4\nDjga+KlSqiTT1+nWwZ22l0bIJh8B58W+rwHyMH/Z/45tew3zDyDrKKVGA2OB12Objia7630c8K7W\nul5rXa61vorsrzPAHpqWKSnGvJkPS/oknk31DgAnY84hijualr/jI4ClWutarXUjsAiYnumLdPfg\n3hdzOYS4+NIIWUVrHZhsq98AAAIRSURBVNFaN8R+vBJ4A8hL+mi+G+h3QArX9e6iaZ0iyP56DwVy\nlVL/VkotUEodS/bXGa31PGCwUmo9ZmPmZ0B10iFZU2+tdTgWrJOl+x03j2/79B509+De3H6tT/Nt\np5Q6AzO4X9dsV1bWWyl1KbBEa72plUOysd4WzBbs2ZipisdJrWc21hml1Bxgq9Z6JHAM5uKDybKy\n3q1ora779B509+De1tIIWUUpdQLwK+AkrXUt4E2aGTyA1I942eIU4Ayl1MfA94Ffk/31rsCcDBjW\nWm8A6oH6LK8zmOmGtwG01iuAHKB30v5srXdcur/r5vFtn96D7h7c21oaIWsopQqBO4FTtdbxjsV3\ngXNi358DvHUgytaVtNYXaK2naK2nAo9gjpbJ9nrPB45RSlljnasesr/OYHYiHgGglBqCeVNbrZSa\nEdt/NtlZ77h0v+NPgClKqaLYaKLpmOt3ZaTbz1BtvjRC7K6fVZRSVwG3AGuTNl+GGfDcwBbgimxe\noE0pdQvmWkZvYw6Zy9p6x5b0uDL24+8xh71me509wGNAH8zhvr/GHAr5EGYj9BOt9Q2tX6H7UEpN\nxuxLGoq5NPoO4LvAEzT7HSulzsV8IJIB3Ke1fibT1+n2wV0IIURL3T0tI4QQIg0J7kIIkYUkuAsh\nRBaS4C6EEFlIgrv4/+3UgQwAAADAIH/re3wFETAkd4AhuQMMBQx58Fx045OkAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "-AGE67k2lGxL",
        "colab_type": "code",
        "outputId": "e1ce36c6-50c0-4b3a-81ea-4fa7861d5e8b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        }
      },
      "cell_type": "code",
      "source": [
        "plt.plot(H.history['loss'])\n",
        "plt.plot(H.history['val_loss'],'r')"
      ],
      "execution_count": 113,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7fd975d12cc0>]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 113
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl4HNWd7vFvLd0ttXbbMt7NZg47\nmC02+5Z9H8gy2SCBkIWZkBmSGQhZJ8lNMhmGhAyZhMyEDIQkd5JcCARCCATC4gDGLDZgH1YbsGRb\n3rSrl6q6f1RLasmyLQvJdqnfz/PwIHVXd58jyW+d+tWpOk4URYiISHK5e7oBIiLy2ijIRUQSTkEu\nIpJwCnIRkYRTkIuIJJy/uz+wra1zzNNkmpqybNnSM57NSYRK7Hcl9hkqs9+V2GfY9X43N9c523su\nUSNy3/f2dBP2iErsdyX2GSqz35XYZxjfficqyEVEZFsKchGRhFOQi4gknIJcRCThFOQiIgmnIBcR\nSTgFuYhIwiUmyDd39PGz3z9NLh/s6aaIiOxVRnVlpzHmKmAREAGXWGuXlh6fDdxYtun+wGXW2l+M\nd0MfXbWB397zPHOnZTnygGnj/fYiIom10yA3xpwGLLDWLjbGHAL8FFgMYK1dC5xe2s4H7gVumYiG\nOm58dWqhGE7E24uIJNZoSitnATcDWGtXAk3GmPoRtjsf+K21tmv8mjfI9+KmFgOtaCQiUm40pZUZ\nwLKy79tKj3UM2+5C4A07e7OmpuyY7jHQ1FANQLYmTXNz3S6/PunU58pRif2uxD7D+PV7LHc/3OYO\nXMaYxcAqa+3wcN/GWO9y1tuTj1+/tZe2ts4xvUdSNTfXqc8VohL7XYl9hl3v945CfzSllRbiEXi/\nWUDrsG3eBtw16haNgefF+49ioBq5iEi50QT5ncC5AMaYY4AWa+3w3cjxwJPj3LYhVCMXERnZToPc\nWrsEWGaMWQJcDVxsjDnfGPPuss1mAhsmqI0A+KUReaARuYjIEKOqkVtrLxv20JPDnj9i3Fq0Hd7A\niFxBLiJSLjFXdvpuf41cpRURkXLJCfL+EXmoEbmISLnEBXmgEbmIyBCJCXJNPxQRGVliglzTD0VE\nRpacIHc1/VBEZCSJCfKB6YehRuQiIuUSE+S+auQiIiNKUJBr1oqIyEgSFOSlhSU0IhcRGSIxQe65\n/SNyBbmISLnEBLnrOriuo+mHIiLDJCbIIa6T62SniMhQiQrylKcRuYjIcIkKct93CXTTLBGRIZIV\n5CqtiIhsI4FBrtKKiEi5xAW5ph+KiAyVqCBP+RqRi4gMl6gg9z1HKwSJiAyTsCB3da8VEZFhkhXk\nvksQRoSRwlxEpF+yglx3QBQR2UYig1xzyUVEBiUqyFN+aUSuVYJERAYkKsg1IhcR2VbCglzLvYmI\nDJewINfJThGR4ZIV5L5KKyIiwyUqyFMDNXKNyEVE+vmj2cgYcxWwCIiAS6y1S8uemwv8EkgDj1lr\nPzkRDYXBWSu6TF9EZNBOR+TGmNOABdbaxcAFwNXDNrkSuNJaewIQGGPmjX8zY6qRi4hsazSllbOA\nmwGstSuBJmNMPYAxxgVOAW4pPX+xtfblCWqrauQiIiMYTWllBrCs7Pu20mMdQDPQCVxljDkGuN9a\ne/mO3qypKYvve2NrbGlEXlNbRXNz3ZjeI6kqrb9QmX2Gyux3JfYZxq/fo6qRD+MM+3o28H1gNXCb\nMeat1trbtvfiLVt6xvCRsf4g37y5m7a2zjG/T9I0N9dVVH+hMvsMldnvSuwz7Hq/dxT6oymttBCP\nwPvNAlpLX28E1lhrX7DWBsDdwGGjbtkuSvVfEKRL9EVEBowmyO8EzgUolU9arLWdANbaIvCiMWZB\nadtjATsRDQXVyEVERrLT0oq1dokxZpkxZgkQAhcbY84H2q21NwGfBX5WOvG5Arh1whqre62IiGxj\nVDVya+1lwx56suy554GTx7NR26PphyIi20rUlZ0qrYiIbCtRQa5L9EVEtpWoIB8oregSfRGRAckK\ncr//fuQakYuI9EtWkGvWiojINhIZ5Jq1IiIyKFFBntKsFRGRbSQqyAdKK7pEX0RkQDKDXCNyEZEB\niQzyQEEuIjIgWUGu6YciIttIVJCnVFoREdlGooJ88MpOjchFRPolK8g1/VBEZBuJCnLPVY1cRGS4\nRAW54zj4nqNZKyIiZRIV5ACe52pELiJSJnFB7rsORd3GVkRkQPKCXCNyEZEhEhjkqpGLiJRLXJDH\nNXIFuYhIv8QFuUorIiJDJS/IXUdrdoqIlElckGv6oYjIUIkLct9zKBZDokhhLiICiQxylwgIFeQi\nIkACg9zzdL8VEZFyiQty39UqQSIi5ZIX5BqRi4gMkcAg1z3JRUTK+aPZyBhzFbAIiIBLrLVLy55b\nDbwCBKWHPmitXTu+zRw0UCPXKkEiIsAogtwYcxqwwFq72BhzCPBTYPGwzd5sre2aiAYON7Dcm0bk\nIiLA6EorZwE3A1hrVwJNxpj6CW3VDvSf7FSNXEQkNprSygxgWdn3baXHOsoe+5ExZl/gAeBya+12\nU7apKYvve2NoaqyuLhP/v76K5ua6Mb9P0lRSX/tVYp+hMvtdiX2G8ev3qGrkwzjDvv8ycAewmXjk\nfg7wm+29eMuWnjF8ZKy5uY58rgjAxo3dNFaNpfnJ09xcR1tb555uxm5ViX2Gyux3JfYZdr3fOwr9\n0SRhC/EIvN8soLX/G2vt9f1fG2NuB45gB0H+Wg1OP1SNXEQERlcjvxM4F8AYcwzQYq3tLH3fYIz5\nozEmXdr2NOCpCWlpidc//VB3QBQRAUYxIrfWLjHGLDPGLAFC4GJjzPlAu7X2ptIo/CFjTC/wOBM4\nGgddECQiMtyoiszW2suGPfRk2XPfB74/no3aEV2iLyIyVAKv7NSIXESkXOKC3NMl+iIiQyQuyPtH\n5IEu0RcRARIZ5BqRi4iUS1yQe7pEX0RkiMQF+UBpRSNyEREgkUGu0oqISLkEBrmmH4qIlEtckOsS\nfRGRoRIX5IM1co3IRUQgiUHuqkYuIlIucUHuqUYuIjJE4oJca3aKiAyV2CAv6hJ9EREggUHuaYUg\nEZEhEhfkqYHSikbkIiKQwCDXmp0iIkMlLsg9TT8UERkicUHuug6u4+hkp4hISeKCHOLyiqYfiojE\nEhnknufqgiARkZJEBrnvOaqRi4iUJDTIXU0/FBEpSWSQe66j29iKiJQkMsh91chFRAYkNMg1a0VE\npF8ig1yzVkREBiUyyDVrRURkUDKD3HUJwogo0qhcRCSZQd6/bqcu0xcRGV2QG2OuMsb81RizxBhz\n/Ha2+ZYx5t5xbV0Z/5GH4aijcF99Bc/TjbNERPrtNMiNMacBC6y1i4ELgKtH2OZQ4NTxb94g/5mn\nYPlyUn99cHCVIJ3wFBEZ1Yj8LOBmAGvtSqDJGFM/bJsrgSvGuW1DhLNmAeC2tgyWVjQiFxEZVZDP\nANrKvm8rPQaAMeZ84C/A6vFs2HDBzNkAeC1ry+5JrhG5iIg/htc4/V8YY6YAHwXOBmaP5sVNTVl8\n39v1Tz3qYACqN22gtiYNQH1jNc3Tanf9vRKoubluTzdht6vEPkNl9rsS+wzj1+/RBHkLZSNwYBbQ\nWvr6TKAZuB/IAAcYY66y1v7D9t5sy5aesbU0StFcVUXhpTUUCwEAG9q6SFXAFMTm5jra2jr3dDN2\nq0rsM1Rmvyuxz7Dr/d5R6I+mtHIncC6AMeYYoMVa2wlgrf2NtfZQa+0i4N3AYzsK8dfEcWDOnLi0\nohq5iMiAnQa5tXYJsMwYs4R4xsrFxpjzjTHvnvDWDTdnDu7GNtJhEVCNXEQERlkjt9ZeNuyhJ0fY\nZjVw+mtv0g7MmQNAQ/tGQPPIRUQgaVd2zp0LQP2WeBKNSisiIkkL8tKIvG7zOgAKKq2IiCQ1yDUi\nFxHpl8ggr928HoCibpolIpKwIC/VyLObSkFe1IhcRCRZQT5tGlE6TW0pyNu783u4QSIie16ygtxx\nCGfOomZjHOQtm7r3cINERPa8ZAU5EMyajb+pjUxUpHWjglxEJHFBHs6chRNFLPD7aNnUreXeRKTi\nJS/IZ8czVxbQSW8uYGuX6uQiUtkSF+RBaYGJ+cV2AFpUXhGRCpe4IA9LC0zM6tsCKMhFRJIX5KUR\n+dTOTYBmroiIJC7Ig1mly/S3bMBxYK1G5CJS4RIX5NG0aUSpFH7rWqY3ZWndqJkrIlLZEhfkuC7h\nzFm4LS3Mmpqlu69Ih67wFJEKlrwgJ55L7q5fx5zGeBFmnfAUkUqWyCAPZsUXBe3n9gLQsmmMCzqL\niEwCiQzysHTCc25eUxBFRBIa5PEUxOlb1+OgIBeRypbIIC+8bjEANbfdQnNjteaSi0hFS2SQF488\nmsLhR5L+0x0clOqjs6dAR49mrohIZUpkkAP0ffAjOEHAqSvuBtAtbUWkYiU2yHPnvIeoqooj/nIL\nRBGvbOja000SEdkjEhvkUWMTube+g9q1qzls7TM8/tzGPd0kEZE9IrFBDtD3ofMAOOeFe1n18hat\n4SkiFSnRQV448WSK++3PwhX3ke3t4tFVG/Z0k0REdrtEBzmOQ98HPoyfz/H6p+9m6cr1e7pFIiK7\nXbKDHOj70PmEdfW8/9HfsvaFFrZ05vZ0k0REdqvEB3k0dSo9n/kHaro7OOeR/8dSlVdEpMIkPsgB\nei/6NIWZs3jH47fy3F9X7OnmiIjsVv5oNjLGXAUsAiLgEmvt0rLnPg5cAATAk8DF1trdu9JDdTW9\nl3+J+s98ipNvupZN553J1Iaq3doEEZE9ZacjcmPMacACa+1i4sC+uuy5LPB+4BRr7UnAwcDiCWrr\nDuXe83627m848+l7WPW7u/dEE0RkL+Zs2ED229+g5ptfw12/blSv8Z61ZH51I6n77sV95WUIgtF/\nYLEIvb2wG1YwG82I/CzgZgBr7UpjTJMxpt5a22Gt7Sk93x/qDcDofkLjzfPI/cs3cT90Lmd/8eO0\nz7ie9BvO3iNNEZHx4y99mJpvfZ3eCz5B/q1v3+XXu+taqb7m+1Rffx1Ob7yGQfW1P6T3ox8nd857\ncDZuxF3XitPdRVTfQNTQiNvaQtX/vZHUskeHvFdUXU3h+EXkTzmVwoknE86eQ9g0BRyH9D13k7n1\nZtJ//hPO1q04YRi/JpMh3GcG4T4z6P7cZRTOOOu1/1CGcXa23qUx5lrgNmvt70rf3w9cYK19tmyb\ny4BLgO9Za7+zo/crFoPI973X3PDtefKK73Lot7+AR4R7zX/AJz85YZ8lslsFAbS1QW1t/N9Ey+fh\nu9+FOXPgfe+DqlGUK6MIenoGR6FVVeCPMF7sH9k6zuB/I7nhBrjwwrgtrgs/+hF8/ONDt1m/HpYt\ngyeegMZGOO44OPJIWLoUrrkGfvvbeHQ8dy5cdhl4HnzjG/Dqqzvui+vCG94Ab3tb/BnPPw8rVsBT\nT428bSm4mTMH5s+HTAZSKdi8GVpb49/dN78Jl16648/dvu38kMYW5A8AHysP8tLj1cDtwBettQ9u\n7/3a2jrHfJzR3FxHW1vnDrcJwpAbvnIdF13/FRp6O8i95e10f/5ygsMOH+vH7nGj6fdkM2n73NdH\n1W//l8IJiwgWHLTN00P6HQSk//wnqq6/Dv/xx3A3tuGEIWFtHd1f+hp9530sDpByUQSFQhyMqdTo\n25XPQzo9pJ31F36EzJ13ABBOnUrvhz9K/vVvJJwzl3CfGdt8tr9sKXWf+yz+04MTDsKGRvre97f0\nnXcBwZy5ZG69merrryP1yEODL8xkKBx9DIXXLaaw8Fii6niHkb7vL2R/eDVhfQPdl3+Rmn/7Nu6m\nTXRd8RWCw48g/cc/kL77T3ivvLxNdyLXHRgRFw8+hN6Pf4q+931gsI99fVT94gb8Z54mnDmTcOYs\nwtpa3I4OnPZ2SPnk3v4uwpmztnlvZ8MG0kvux1+2FLetDXfTRpyuLgqLTiT39ndSXHjsyDumKBry\n+K7+jTc3172mIP8q0Gqt/XHp+xeBo6y1ncaYKcDh1tr7Ss/9E4C19l+3934THeQAT7+0mRuu/SNf\nuvtq9l39NAC5t7+L4oEH4q1ZjffqqxQWn0T35y8f+se7l5qsoZa6989k/vB7uv/5CqIpU4c8tzv7\n7D1rIZ8nOPyI1/Q+qfvupeoXN1A84ihy73gX4dx5Q5//813UXXYp3uqXiLJZOr93Dbl3nQOAu34d\n1T/+IdlN6+mLXKJUivRf7sV7eTUAwbz5hDNmEjZPJ/XAfbjtW8kvOpHeiz5N6onHSD3wF/yVzwyU\nDqKqKnrPu4CeSy4lmjZt+40OQ6p/+ANqvvUvFI49nt6//yz5xSfTcP4HSd93D/nTz6R4+JFU3fg/\nuFu2DLwsSqUoHnIYhTPOIn/aGaTvuI3qn/wIJ4rIn3QKUXU1AP7yJ/E2xBfqRdkanJ74LqWFY48j\nqs4CkO5sJ1qxYiB4yxX3P4COn/8vwYEL8J5/job3vBNv7eBIOmxspHDCIopHLaR4xFE4W7eQeuIx\n/OVPEsyZQ9/5F1JYfNL2R/x70O4O8hOBr1lrX2+MOQa42lp7cum5fYC/Akdaa7uMMb8BbugfvY9k\ndwQ5wNW/Wc4Tz7XxpRkbWfjLH5J64vFttiksPIaOn/wP4bz5Y2qP9+LzpO75M33v/yDU1Ox0e//h\nh8j++Bpyb3kbuXPeO+o/rskY5O6a1TSdcRJuV2f8j/UXvybY/8CB55un1dK2cft3tHRb1pJ6aAnF\noxcOeV0/Z+sWMrfcTOaWm4kyaYrHHk/hmOMIDjmUcFozeB7eUyuoufI7ZG67hchx6LnkUno+f/ng\nSDaK4kPynYxsnY0bqf3KF6j69a+GPF44eiHhrDlEVRnczZtJ3/tnIs8j9zfvIX3773G7u+j5xKfB\n86n+6bU4fX1DXh9ls/Sd8176zr+A4hFHDfZ9/TpqL/scmdtuGdzW9ykefChRbS1kMnjPP4e39lWi\nbA29H/ko4Zw5RKk0UXU1gTmY4iGH4XR1Uff3nyBz958Ia+twu+K/sbCmFre7i9yb3krHT34Wlwh6\ne8ncejP+ymdwX30F75U1+E+twMkP3t+oeMCBdF15NYUTTy77IRRI/+H3VP/sv/FeeZncu86h90Pn\nEc7fd2CT5uY6Nr7Ugv/oUvynVkBYKrtUV9P3nvcTNTYN9n3tq9R++QsE8+aTf8ObKBz/upFLNwmw\nW4McwBjzbeBUIAQuBhYC7dbam4wx55ceKxJPP/zUjqYf7q4gX7+5hy/99yOkfZcvn3css55djhMU\nCebvS9TQQO1ln6Pq178ibGgk9+5zcDdtwt2wniidJlhwEMWDDqZ46OEUj144Ym0wteQB6s/7AG77\nVoL5+9L5vWsonHTKiG1xNm2i5utfpvoXNww8lj/pFLq+fSWBOfi19zuKIJcbXQ3ztSgWqfrlz8l+\n/9+JqjLkzziLwulnkj/xFCiNwEb7Po3vfDOppQ+TP/Ns0n++i7Cxka5v/ive88+SueN2/LWv0nnF\nV+n76IUDL3M62qn62X+Tue0WUo8/NvB44bgT6DvnvZBK4b34At6zq0jfd++QkCkX+T5h83S81pb4\n9ccci7txE97LqyksPIbeT/4dqQcfIH3XH3FbWwhnzyHY/0DCadNw2zbgrmvF3bKFqKqKqLoad/16\n3I52CkceTfdXv4H30otkbrmJ1AP34ZTNcigc/zo6v/PvBIcfgfespf78D+A//xwAwazZ9Fz6z9S9\n991sat2Mk88TzpxJVFe/3R9j+o9/wH98GYUTFsWrZpUPJnI5qn7+P2Sv+u7AiHjIz8DziKqqcbu7\nyJ95Nh0/+DFu2way//E9Mjf9htw73kXnf1y7451YdzfpJfeTuu9ewuZ96L3oU2P6G5yMA5XR2O1B\nPp52V5AD3L+8hetuX8Xc6bV84cPHkkmVnWSNIjK/upG6yy4dPBwtq6sNbJZOUyzV7/InnUzxhEWk\n//gH6i75NIQhube/k8zvbsIJQ/refQ5EEd5LL8WHf45D5Pu47e04Pd0UDzuC7s9fTtWvfk7mjtuJ\nfJ++v/0wvRf//cCo0mnfSur++4iy1RQXHkvUNGWH/fYfe5S6Sy/Bf3oFYWMj4cxZFI88mq6vf2vI\nSIYgiEsIvh8f9kYR/rOr8J5+Cm/NGsI5cygedDDBvPl4a1bjP/UkvrWEU6YQ7Ls/0ZQpVF/7Q/yV\nzxBl40Nip6cHgHBaMz2f/Dv6PnYhUW0d5PP4q57BbdswcNIrbJ4ejypdl+yV36HmO9+k711/Q+eP\nr4t/D5d+BqdYjH/mmQxOJgMdHfR+5GN0/Z9/JfP731Hz5S/gbVhP5PsUTjyF/Cmnkn7wflJ/uQdn\n2N9x8eBD6Dv3/eTOfS+RnyL12KOkli3FXf0SXsta3NYWgrnz6LnkUgpnnIXT1Unt5Z+n6n9/OfAe\nYVMTwUEH4768ZiD0AcIpUwinTMXJ53F6eog8j96/u4TeCz4xdHSYy+H0dOPkclAsEs6eM+QozOns\nIPvdbxPMm0ffh86HqqrxD7WeHlIPPYjT24dTyON0dOA/8xT+iuW4LWvp/dhF9F78maH17p6eeMe8\nm8oRCvJRb1+ZQQ5w/R2ruPeJFhYftg8Xvu1QnGF/nM6GDXjrWwmmz4hriX19+M8/i2dX4S9/gtTD\nD+E/tXxgZBV5Hk4QENbV03Hdzymcenocpp+9GH/VynibqiqCWbPBdXGKRSLPo+/8C4b8Q0//8Q/U\nfPly/JdeJHIc8m98C053N6m/PjAQaBAfrvpnnE77iadTOPU0ovoGCEOczZvJfv9Kqn/ynzhhSOH4\n1+F0tOO2tOB2dlA84EA6bvxfgv0PxFu1krrPfHLE8tKuiEo3Keu57IuEjU2klj5M+s47qLrxetzO\nDsLGRoJ5++KvfBqnUNjm9eG0ZvKnnkbmdzcRzpjJlnseHNjZ+A8/ROb2Wykc/zryp59Js5un+Na3\nxzuoadNwN24kqqqi57Ofo/djHx96uN3aQvqO24myWYL9DiDYb3+i5uYx9TF9x+34Ty0nf8rpFI87\nPp7hANDVhbtlM2Hz9Ak98qnEUKvEPoOCfJdeUyiGfOcXj/FiSwfnnn4Ab1k0hnp4VxepRx8hteQB\n0g/eD2FI57//gOCQQwe3yefj0Jm+T3yme/hsgpEEAenbbiH7g++RejIO2cLRC8m/4c1QLJBa9ij+\nY8twOzuAuCQQNTbibNkysGMp7n8AXf/+g8G6ZBhS882vkf3BVYSNjfS97wNUX/dfOPk8uTe9lbB5\nOk5vD4QBwYEHUTz0cIJ998Nb+wqetXgvryaYO5/iEUcSHHwIztatcbli7SvkTzxlxNk/TvtWqn/6\nE6p/fA1OVxfFww6neORCgrlz6Z8x5b3wHJm77sRt20DkOLTfdNvQWuowzc11tK1eR91nL6bqd/+P\n3Fmvp+tb/0a4736j+IUlVyWGWiX2GRTku/y6zR19fP36R2nvynP2cXN4/5kLcN296Cx2FOE9/RTR\n1KnbTncKAprXWLp/czPpe+7Gad9KNGUq4ZSpFBceQ8+n/n7EEWLmlz+n7nOX4BQKBNP3oevKq8m/\n8c0T248giOfSbq+uGob4Tz4OQUDxuBN2+FYDv+sowl376jZlicmqEkOtEvsMCvIxvXbj1l6u+vWT\ntG7qYeGCaVz0jsOG1sz3YmPtt//Iw6TvuYveiz5F1DRlAlo2cfSPu3JUYp9hfIN8Utz9cDSmNVZz\nxYeP5ZD5TTz+3Ea+feNjbO7o2/kLE6x4wuvo+ecrEhfiIrJrKibIAbJVKf7hvUdx8pEzWbOuk3/5\n2VLsy1t2/kIRkb1YRQU5gO+5fPTNB/PB1x9Ed1+Rf/vVE9z219XkCrtwVzMRkb1IxQU5gOM4nHXs\nHD73/qPJVvn89i8v8k//uYTbH1pDb6648zcQEdmLVGSQ9zPzmvjmxxfx9hP3pRhE/ObeF/jnH/2V\nu5e9SjHY9r4PIiJ7o4oOcoDa6hTvPnV/vvupxbzrlP0oBiE3/ulZvvhfD/PwM+sJRriRj4jI3iSZ\nd5uZANmqFO84aT9OXzibWx9Yzb1PrOXHtzzNb+6t4uzj5nDKkbPIVunHJSJ7HyXTMPXZNB98w0Gc\nffwc7nzkFR5c0cr//fPz3HTfixx54DROOHg6Rx4wlXRC5qCLyOSnIN+OfZqyfPiNhnefuj9/eWIt\nD6xYx6OrNvDoqg1kUh5HHTiV48x0jjhgamIuLBKRyUlBvhO11Sneunhf3rJoPq9s6OKRlRtYumo9\nj6zcwCMrN5D2XQ6e38RRB0zliAOmMq1hF27nKiIyDhTko+Q4DvP2qWPePnWcc9r+vLy+i0ftBh57\nto3lL2xi+QubAJgxJcvh+03h0P2msP+seuqze/8KRCKSbAryMXAch/kz6pg/o45zTjuAjVt7WfFi\nHOarXt7KXcte5a5l8XJUTXUZ9p1Rx6H7TuGI/acwvSm7h1svIpONgnwcTGus5oxj5nDGMXMoBiEv\nrG1n5ZotrF7XyZp1nTz+3EYef24jAPs0VbNgbiP7z6xnv5n1zJpWQ8qv+FmgIvIaKMjHme+5mHlN\nmHmDCx9sbO/lqRc3s+LFTTyzZgsPLG/lgeWtQHxn1umN1cycWsM+U6ppboz/mzEly9SGKtwKuHWr\niLw2CvLdYFpDNacvnM3pC2cThCGtG3t4sbWD1es6aWnromVTD088v3Gb12XSHrOn1XDg3Caa6zPM\n26eWmVNrqKnyt1npSEQql4J8N/NclznTa5kzvZZTBxdGp6MnT9uWXtq29rJhay+tm3pY29bFmnWd\nvNjSMeQ9MimPKfUZmhurmTk1y6ypNUxrqKIq41OV9qjLpqmt3vHK7yIyeSjI9xL12TT12TQHzG4Y\n8ngxCMlFDk+uWsfL67vYsKWXzR19bOroo3VTz8BsmeGa6jLMnV7L7Gk1NNVlaKrL0FiXYUpdFQ21\naZVsRCYRBflezvdcZjbXUeM7nDhsucyu3gKtm7pp3dTD5o4++vIBuULAls4cr2zoGjItspznOjTW\nxuHeUJumoSaN77m4roPvOezTlGXu9FpmTavB93QiVmRvpyBPsNrqFAvmNLJgTuOIz3f25Gnd1MPW\nrhxbO3Ns6cqxuSM3MKJ/saUKQvanAAAKDElEQVSDcAdL/cWBn6a+JkNDTZr6mhS11XHZpi6boi4b\nP1ad8Un7HpmUSybt4Y1m4WkRGTcK8kmsLpumbgcXJIVhRGdvgY7uPEEYEoZQKAa0bOzm5Q1dvNrW\nxdbOHK9s6OSlYPRLraZ9l6qMT02VT302TV02RUNthqn1VUypj3cKKd8rbefRVJdR+Iu8BgryCua6\nDg01cWmlXPnUSYAoiujuK9LZk6ert0BnT4Gu0g6gs6dAb75IvhCQL4T05Yv05gP6ckU6ewqs29TD\nznYBnuswpT5DU10V6ZRLynOpq81AGFGd8ahO+2SrfGqqU9RUpahKe2RSXunEbnxEoFk8UskU5LJT\njuNQW50a00yYIAzp6i2ytXOwpNPZU6BQDCkUQ7r7Cmxs76Ntay/PvrJ1TO1L+y71NWkaazM01qZp\nqM1QlfZIeS4p38X3XTIpj5Qf7yR8z8X3HKoyPvU1aRqyaTLp+MZnUanUpB2DJImCXCaU57oDo/75\nM+p2uG0YRhSCOODr6qtpWddOby6gN1+kp69IV2+B7t4CuUJ8UrcvH9DRnae9O097V26nNf8dcWDg\nyKH/3EBDbYb6bJrqjE8241OV8UinPDK+Szrlka2KH6/O+PFOwndJ+x6+5+D7Lr7rkkq5miEkE05B\nLnsN13XIuHHZZFpjNVFh19ZPDcOIzp442PvyAcXSTqH/v1wxoFgMKQYRxSCkJ1ekoztPR3eevkKA\n6zi4DuQKIe3dOdas6yQIx7ZjKJdOxUcE1ek49KszHlXpeM5/OhUfOThuvAOZ2pjFJaIumyZb5ZNJ\nxT8P33NwXQfHceJlvRxwiB/zSrONUr6n2z1UKAW5TBqu69BQm6GhNjMu7xdGEb25Yum/gN5ckUIx\nJF86IujNFenJxUcLhWJIIQjJF0KCMBzyfa4QkMvHRxbt3XlyhWBc2jeS/vMG/TuCmqoU1WmPIIwo\nBhFRFJHy49lF8Q7CLe0IXGqqfRpq4hPkKd+NdxoOpVJU/xGHO/Cc7D0U5CLb4ToONVXxCdbxFIQh\nuXxYKg8VCYKIIIwIo4h0VYpXWtvjk8i5IrlCQD4f7xSiKCKK4h1MXEGKCCMIgvgoI1cI4pPQPflx\nO5oYiePEVxd7rkMUxSUp33MGykzVpSuMM+l4RxGW+uaWzrXUVA/uXIIwIptNExaDgdenUx7pVFym\n8kpHHK7rEIYRxTAiDKPSEUipxJXxK/56h1EFuTHmKmAR8e/sEmvt0rLnzgC+BQSABS601mrFYpHt\n8FyXbJVbWgN26NFDc3Mds5te++IkURSRL8Qnk3vzAb7r4HkOruOQL4bkShePBUFIUDo3UT4TKd5x\nxOWqIIx3FOVHI7l8QBBFxONyh2IQ0psrsrkzR6G4+//5ZzM+ddkU6ZSH6zg4pSOJ/nMX5ecpHAe8\n0glv13EGdoyOw2DJq/Qax4mP9Pqny/bvYPr/n+r/3ncHdjhe//ap+LHdcfSy0yA3xpwGLLDWLjbG\nHAL8FFhctsm1wBnW2leNMb8G3gTcPiGtFZFRcRwnLp+kd/8yhMWgdLSRi89T9AdcEEZ09xXo6inQ\nlw/i0bbn0thYzboNnfT0xaWqfCEgXwwpFEKCKCIIQsIwikPSc/Ech2IYl63yxYDu0pTYzp48HT2F\nOJhLo/2JOioZLddxcEsnNTzX4T1nHMCZx8wZ988ZzYj8LOBmAGvtSmNMkzGm3lrbfyenY8u+bgOm\njnsrRSQx+mvqI5Wkmtn2aKO5uY62qROz4Er/+Yqw7CAhjEohH4QDJR/HiUs3faVyV75QKmURH5Xk\nCyGFYnw0ki+dPO/f4eQLQekzotKOp3T0UoyvrSgf8TeN0/mb4UYT5DOAZWXft5Ue6wDoD3FjzEzg\nDcCXdvRmTU1ZfH/so4Tm5h1PYZusKrHfldhnqMx+V2KfYfz6PZaTndsUfIwx04FbgU9ba0e+HV/J\nli09Y/jIWHNzHW1tnWN+fVJVYr8rsc9Qmf2uxD7Drvd7R6E/miBvIR6B95sFtPZ/Y4ypB/4AXGGt\nvXPUrRIRkXExmjk7dwLnAhhjjgFarLXlu5ErgaustXdMQPtERGQndjoit9YuMcYsM8YsAULgYmPM\n+UA78EfgI8ACY8yFpZf8wlp77UQ1WEREhhpVjdxae9mwh54s+3piTsOKiMioVPblUCIik4CCXEQk\n4RTkIiIJ50RjvH+ziIjsHTQiFxFJOAW5iEjCKchFRBJOQS4iknAKchGRhFOQi4gknIJcRCThErP4\n8o7WDZ1sjDH/CpxC/Pv5FrAUuAHwiG8h/GFrbW7PtXBiGGOqgaeArwN3Uxl9/iDwT0AR+DKwnEnc\nb2NMLXA90ER8n6avAeuA/yT+t73cWvupPdfC8WWMORz4HfEdYv/DGDOXEX6/pb+DzxLfmPBaa+1/\n78rnJGJEXr5uKHABcPUebtKEKS1mfXipr28Cvgf8C3CNtfYU4HngY3uwiRPpi8Dm0teTvs/GmKnA\nV4CTgbcB72Ty9/t8wFprzyC+Pfb3if/GL7HWngQ0GGPevAfbN26MMTXAD4gHJf22+f2WtvsycDZw\nOvAPxpgpu/JZiQhyhq0bCjSVFrSYjO4D3lP6eitQQ/zLvaX02K3Ev/BJxRhzMHAocFvpodOZ5H0m\n7tNd1tpOa22rtfYiJn+/NzK4rm8T8Y57v7Ij7MnU5xzwFuLFefqdzra/39cBS6217dbaXuBB4KRd\n+aCkBPkM4rVC+/WvGzrpWGsDa2136dsLgNuBmrLD6w3AzD3SuIl1JfCPZd9XQp/3BbLGmFuMMfcb\nY85ikvfbWvsrYJ4x5nniQcvngC1lm0yaPltri6VgLjfS73d4vu3yzyApQT7cNuuGTjbGmHcSB/nf\nDXtq0vXdGPMR4K/W2pe2s8mk63OJQzw6/RviksN1DO3rpOu3MeZDwMvW2gOBM4GfD9tk0vV5B7bX\n113+GSQlyHe4buhkY4x5I3AF8GZrbTvQVToRCDCboYdqk8FbgXcaYx4CLgS+xOTvM8B6YElp5PYC\n0Al0TvJ+n0S8shjW2ieBamBa2fOTsc/lRvq7Hp5vu/wzSEqQ72zd0EnDGNMAfBd4m7W2/8TfXcA5\npa/PASbV+qjW2vdZa4+31i4C/ot41sqk7nPJncCZxhi3dOKzlsnf7+eJa8IYY+YT77xWGmNOLj3/\nN0y+Ppcb6ff7MHC8MaaxNKvnJOD+XXnTxNzG1hjzbeBUSuuGlvbmk44x5iLgq8CzZQ+fRxxwVcAa\n4KPW2sLub93EM8Z8FVhNPGq7nkneZ2PMJ4hLaADfIJ5qOmn7XQqqnwL7EE+v/RLx9MMfEw8sH7bW\n/uP23yE5jDHHEp/72RcoAGuBDwI/Y9jv1xhzLvB54imYP7DW3rgrn5WYIBcRkZElpbQiIiLboSAX\nEUk4BbmISMIpyEVEEk5BLiKScApyEZGEU5CLiCTc/wc11nIwNRDsJgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "4RIGVSojmy-R",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Predict"
      ]
    },
    {
      "metadata": {
        "id": "tnigE74rm39a",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "unlabeled_images_test = pd.read_csv('gdrive/My Drive/dataML/test.csv')\n",
        "#unlabeled_images_test = pd.read_csv('test.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "3Fotf1KrpXVE",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "X_unlabeled = unlabeled_images_test.values.reshape(unlabeled_images_test.shape[0],28,28,1)/255"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "jeRnCHzdutQ0",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "y_pred = model.predict(X_unlabeled)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "3M_fePteu-3X",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "y_label = np.argmax(y_pred, axis=1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "RX5PuSUmvRri",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Save csv"
      ]
    },
    {
      "metadata": {
        "id": "zU5Q1fSRvVbn",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "imageId = np.arange(1,y_label.shape[0]+1).tolist()\n",
        "prediction_pd = pd.DataFrame({'ImageId':imageId, 'Label':y_label})\n",
        "prediction_pd.to_csv('gdrive/My Drive/dataML/out_cnn05.csv',sep = ',', index = False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "_qetEX7AgBQY",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Tensorflow"
      ]
    },
    {
      "metadata": {
        "id": "4keUL7d0gBQZ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Helper functions for batch learning"
      ]
    },
    {
      "metadata": {
        "id": "czdbjPfcgBQd",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def one_hot_encode(vec, vals=10):\n",
        "    '''\n",
        "    For use to one-hot encode the 10- possible labels\n",
        "    '''\n",
        "    n = len(vec)\n",
        "    out = np.zeros((n, vals))\n",
        "    out[range(n), vec] = 1\n",
        "    return out"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Afc2XmK2gBQh",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class CifarHelper():\n",
        "    \n",
        "    def __init__(self):\n",
        "        self.i = 0\n",
        "        \n",
        "        # Intialize some empty variables for later on\n",
        "        self.training_images = None\n",
        "        self.training_labels = None\n",
        "        \n",
        "        self.test_images = None\n",
        "        self.test_labels = None\n",
        "    \n",
        "    def set_up_images(self):\n",
        "        \n",
        "        print(\"Setting Up Training Images and Labels\")\n",
        "        \n",
        "        # Vertically stacks the training images\n",
        "        self.training_images = train_images.as_matrix()\n",
        "        train_len = self.training_images.shape[0]\n",
        "        \n",
        "        # Reshapes and normalizes training images\n",
        "        self.training_images = self.training_images.reshape(train_len,28,28,1)/255\n",
        "        # One hot Encodes the training labels (e.g. [0,0,0,1,0,0,0,0,0,0])\n",
        "        self.training_labels = one_hot_encode(train_labels.as_matrix().reshape(-1), 10)\n",
        "        \n",
        "        print(\"Setting Up Test Images and Labels\")\n",
        "        \n",
        "        # Vertically stacks the test images\n",
        "        self.test_images = test_images.as_matrix()\n",
        "        test_len = self.test_images.shape[0]\n",
        "        \n",
        "        # Reshapes and normalizes test images\n",
        "        self.test_images = self.test_images.reshape(test_len,28,28,1)/255\n",
        "        # One hot Encodes the test labels (e.g. [0,0,0,1,0,0,0,0,0,0])\n",
        "        self.test_labels = one_hot_encode(test_labels.as_matrix().reshape(-1), 10)\n",
        "\n",
        "        \n",
        "    def next_batch(self, batch_size):\n",
        "        # Note that the 100 dimension in the reshape call is set by an assumed batch size of 100\n",
        "        x = self.training_images[self.i:self.i+batch_size]\n",
        "        y = self.training_labels[self.i:self.i+batch_size]\n",
        "        self.i = (self.i + batch_size) % len(self.training_images)\n",
        "        return x, y"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "1qAiPT1-gBQp",
        "colab_type": "code",
        "outputId": "8aaf4220-78b9-481a-d9af-dee6c7e229d3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "# Before Your tf.Session run these two lines\n",
        "ch = CifarHelper()\n",
        "ch.set_up_images()\n",
        "\n",
        "# During your session to grab the next batch use this line\n",
        "# (Just like we did for mnist.train.next_batch)\n",
        "# batch = ch.next_batch(100)"
      ],
      "execution_count": 207,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Setting Up Training Images and Labels\n",
            "Setting Up Test Images and Labels\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "QUkUv8sKgBQw",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Creating the Model\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "l1XSMzIpgBQx",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "** Create 2 placeholders, x and y_true. Their shapes should be: **\n",
        "\n",
        "* X shape = [None,28,28,1]\n",
        "* Y_true shape = [None,10]"
      ]
    },
    {
      "metadata": {
        "id": "8Y_4DDQvgBQz",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "X = tf.placeholder(tf.float32, shape=[None,28,28,1])\n",
        "Y_true = tf.placeholder(tf.float32, shape=[None,10])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "5egQVsS4NhxW",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Initialize W and b"
      ]
    },
    {
      "metadata": {
        "id": "xiVQdlDsN1W-",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "W1 = tf.Variable(tf.truncated_normal([784, 10], stddev=0.1))  # 784 = 28 * 28\n",
        "B1 = tf.Variable(tf.ones([10])/10)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "u4RDJq8pO_Og",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### layers nn"
      ]
    },
    {
      "metadata": {
        "id": "x52yDbi6bCMb",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "XX = tf.reshape(X,[-1,784])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "DM5_O098O4Di",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "Ylogits = tf.matmul(XX, W1) + B1\n",
        "Y = tf.nn.softmax(Ylogits)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "tT4TvNz-gBRI",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Loss Function"
      ]
    },
    {
      "metadata": {
        "id": "wkqQ5GurgBRJ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "cross_entropy = -tf.reduce_mean(Y_true * tf.log(Y)) * 1000.0 \n",
        "#cross_entropy = tf.losses.softmax_cross_entropy(onehot_labels = Y_true, logits = Ylogits)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "jmnEUVWxgBRM",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Optimizer"
      ]
    },
    {
      "metadata": {
        "id": "MoyIlzCagBRN",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "#optimizer = tf.train.GradientDescentOptimizer(learning_rate = 0.005)\n",
        "optimizer = tf.train.AdamOptimizer(learning_rate=0.001)\n",
        "train = optimizer.minimize(cross_entropy)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "47rAzeVNgBRP",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Intialize Variables"
      ]
    },
    {
      "metadata": {
        "id": "7WG1AszIgBRQ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "init = tf.global_variables_initializer()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "rseSYLjggBRb",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Saving the Model"
      ]
    },
    {
      "metadata": {
        "id": "OwdUgOG4gBRc",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "saver = tf.train.Saver()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "xGdHTpE0gBRe",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Graph Session\n",
        "\n",
        "** Perform the training and test print outs in a Tf session and run your model! **"
      ]
    },
    {
      "metadata": {
        "id": "pQHEYbyZgBRf",
        "colab_type": "code",
        "outputId": "34cf9e7f-46b5-4587-dc12-502a30b7f97d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 15337
        }
      },
      "cell_type": "code",
      "source": [
        "history = {'acc_train':list(),'acc_val':list(),\n",
        "           'loss_train':list(),'loss_val':list()}\n",
        "with tf.Session() as sess:\n",
        "    sess.run(tf.global_variables_initializer())\n",
        "    \n",
        "    for i in range(30000):\n",
        "        batch = ch.next_batch(100)\n",
        "        sess.run(train, feed_dict={X: batch[0], Y_true: batch[1]})\n",
        "        \n",
        "        # PRINT OUT A MESSAGE EVERY 100 STEPS\n",
        "        if i%100 == 0:\n",
        "            \n",
        "            # Test the Train Model\n",
        "            feed_dict_train = {X: batch[0], Y_true: batch[1]}\n",
        "            feed_dict_val = {X:ch.test_images, Y_true:ch.test_labels}\n",
        "\n",
        "            matches = tf.equal(tf.argmax(Y,1),tf.argmax(Y_true,1))\n",
        "            acc = tf.reduce_mean(tf.cast(matches,tf.float32))\n",
        "            history['acc_train'].append(sess.run(acc, feed_dict = feed_dict_train))\n",
        "            history['acc_val'].append(sess.run(acc, feed_dict = feed_dict_val))\n",
        "\n",
        "            history['loss_train'].append(sess.run(cross_entropy, feed_dict = feed_dict_train))\n",
        "            history['loss_val'].append(sess.run(cross_entropy, feed_dict = feed_dict_val))\n",
        "            \n",
        "            print(\"Iteration {}:\\tloss_train={:.6f}:\\tloss_val={:.6f}:\\tacc_train={:.6f}:\\tacc_val={:.6f}\"\n",
        "                  .format(i,history['loss_train'][-1],history['loss_val'][-1],history['acc_train'][-1],history['acc_val'][-1]))\n",
        "            \n",
        "            print('\\n')\n",
        "        \n",
        "    saver.save(sess,'models_saving/my_model.ckpt')"
      ],
      "execution_count": 253,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Iteration 0:\tloss_train=230.272629:\tloss_val=230.249054:\tacc_train=0.070000:\tacc_val=0.111905\n",
            "\n",
            "\n",
            "Iteration 100:\tloss_train=228.779739:\tloss_val=228.786118:\tacc_train=0.270000:\tacc_val=0.240476\n",
            "\n",
            "\n",
            "Iteration 200:\tloss_train=227.326981:\tloss_val=227.170975:\tacc_train=0.370000:\tacc_val=0.438095\n",
            "\n",
            "\n",
            "Iteration 300:\tloss_train=225.896072:\tloss_val=225.760147:\tacc_train=0.430000:\tacc_val=0.478571\n",
            "\n",
            "\n",
            "Iteration 400:\tloss_train=223.582092:\tloss_val=224.205093:\tacc_train=0.570000:\tacc_val=0.502381\n",
            "\n",
            "\n",
            "Iteration 500:\tloss_train=221.488022:\tloss_val=222.739929:\tacc_train=0.560000:\tacc_val=0.521429\n",
            "\n",
            "\n",
            "Iteration 600:\tloss_train=222.403641:\tloss_val=221.132553:\tacc_train=0.530000:\tacc_val=0.561905\n",
            "\n",
            "\n",
            "Iteration 700:\tloss_train=219.779999:\tloss_val=219.747086:\tacc_train=0.600000:\tacc_val=0.569048\n",
            "\n",
            "\n",
            "Iteration 800:\tloss_train=217.090851:\tloss_val=218.214050:\tacc_train=0.630000:\tacc_val=0.578571\n",
            "\n",
            "\n",
            "Iteration 900:\tloss_train=216.813965:\tloss_val=216.757675:\tacc_train=0.580000:\tacc_val=0.621429\n",
            "\n",
            "\n",
            "Iteration 1000:\tloss_train=214.546585:\tloss_val=215.229904:\tacc_train=0.650000:\tacc_val=0.628571\n",
            "\n",
            "\n",
            "Iteration 1100:\tloss_train=213.214813:\tloss_val=213.840652:\tacc_train=0.660000:\tacc_val=0.635714\n",
            "\n",
            "\n",
            "Iteration 1200:\tloss_train=213.010529:\tloss_val=212.369843:\tacc_train=0.600000:\tacc_val=0.623810\n",
            "\n",
            "\n",
            "Iteration 1300:\tloss_train=209.463562:\tloss_val=210.956100:\tacc_train=0.780000:\tacc_val=0.666667\n",
            "\n",
            "\n",
            "Iteration 1400:\tloss_train=209.903839:\tloss_val=209.491714:\tacc_train=0.640000:\tacc_val=0.650000\n",
            "\n",
            "\n",
            "Iteration 1500:\tloss_train=207.080719:\tloss_val=208.096649:\tacc_train=0.660000:\tacc_val=0.671429\n",
            "\n",
            "\n",
            "Iteration 1600:\tloss_train=204.438782:\tloss_val=206.698364:\tacc_train=0.720000:\tacc_val=0.659524\n",
            "\n",
            "\n",
            "Iteration 1700:\tloss_train=204.436615:\tloss_val=205.318985:\tacc_train=0.680000:\tacc_val=0.676190\n",
            "\n",
            "\n",
            "Iteration 1800:\tloss_train=203.331482:\tloss_val=203.929245:\tacc_train=0.710000:\tacc_val=0.685714\n",
            "\n",
            "\n",
            "Iteration 1900:\tloss_train=201.586639:\tloss_val=202.500885:\tacc_train=0.690000:\tacc_val=0.695238\n",
            "\n",
            "\n",
            "Iteration 2000:\tloss_train=199.304077:\tloss_val=201.187393:\tacc_train=0.780000:\tacc_val=0.688095\n",
            "\n",
            "\n",
            "Iteration 2100:\tloss_train=197.678711:\tloss_val=199.836472:\tacc_train=0.710000:\tacc_val=0.692857\n",
            "\n",
            "\n",
            "Iteration 2200:\tloss_train=198.198120:\tloss_val=198.532288:\tacc_train=0.700000:\tacc_val=0.711905\n",
            "\n",
            "\n",
            "Iteration 2300:\tloss_train=198.226700:\tloss_val=197.098160:\tacc_train=0.650000:\tacc_val=0.721429\n",
            "\n",
            "\n",
            "Iteration 2400:\tloss_train=196.248596:\tloss_val=195.886887:\tacc_train=0.670000:\tacc_val=0.709524\n",
            "\n",
            "\n",
            "Iteration 2500:\tloss_train=196.173615:\tloss_val=194.545486:\tacc_train=0.710000:\tacc_val=0.707143\n",
            "\n",
            "\n",
            "Iteration 2600:\tloss_train=193.130219:\tloss_val=193.302536:\tacc_train=0.750000:\tacc_val=0.728571\n",
            "\n",
            "\n",
            "Iteration 2700:\tloss_train=194.022049:\tloss_val=191.908463:\tacc_train=0.660000:\tacc_val=0.728571\n",
            "\n",
            "\n",
            "Iteration 2800:\tloss_train=192.264099:\tloss_val=190.751297:\tacc_train=0.710000:\tacc_val=0.723810\n",
            "\n",
            "\n",
            "Iteration 2900:\tloss_train=188.023560:\tloss_val=189.447830:\tacc_train=0.680000:\tacc_val=0.719048\n",
            "\n",
            "\n",
            "Iteration 3000:\tloss_train=183.677979:\tloss_val=188.221710:\tacc_train=0.760000:\tacc_val=0.735714\n",
            "\n",
            "\n",
            "Iteration 3100:\tloss_train=185.036255:\tloss_val=186.870224:\tacc_train=0.720000:\tacc_val=0.742857\n",
            "\n",
            "\n",
            "Iteration 3200:\tloss_train=185.666626:\tloss_val=185.745316:\tacc_train=0.760000:\tacc_val=0.735714\n",
            "\n",
            "\n",
            "Iteration 3300:\tloss_train=181.772827:\tloss_val=184.481400:\tacc_train=0.740000:\tacc_val=0.723810\n",
            "\n",
            "\n",
            "Iteration 3400:\tloss_train=176.778992:\tloss_val=183.282364:\tacc_train=0.730000:\tacc_val=0.747619\n",
            "\n",
            "\n",
            "Iteration 3500:\tloss_train=181.644150:\tloss_val=181.987793:\tacc_train=0.780000:\tacc_val=0.745238\n",
            "\n",
            "\n",
            "Iteration 3600:\tloss_train=180.518631:\tloss_val=180.878052:\tacc_train=0.670000:\tacc_val=0.757143\n",
            "\n",
            "\n",
            "Iteration 3700:\tloss_train=175.202454:\tloss_val=179.656647:\tacc_train=0.800000:\tacc_val=0.733333\n",
            "\n",
            "\n",
            "Iteration 3800:\tloss_train=178.122192:\tloss_val=178.497452:\tacc_train=0.810000:\tacc_val=0.750000\n",
            "\n",
            "\n",
            "Iteration 3900:\tloss_train=180.081268:\tloss_val=177.292374:\tacc_train=0.660000:\tacc_val=0.750000\n",
            "\n",
            "\n",
            "Iteration 4000:\tloss_train=173.740204:\tloss_val=176.159149:\tacc_train=0.770000:\tacc_val=0.766667\n",
            "\n",
            "\n",
            "Iteration 4100:\tloss_train=175.377563:\tloss_val=175.014130:\tacc_train=0.710000:\tacc_val=0.750000\n",
            "\n",
            "\n",
            "Iteration 4200:\tloss_train=176.567490:\tloss_val=173.888840:\tacc_train=0.780000:\tacc_val=0.757143\n",
            "\n",
            "\n",
            "Iteration 4300:\tloss_train=176.858780:\tloss_val=172.731308:\tacc_train=0.720000:\tacc_val=0.764286\n",
            "\n",
            "\n",
            "Iteration 4400:\tloss_train=171.444595:\tloss_val=171.587814:\tacc_train=0.710000:\tacc_val=0.776190\n",
            "\n",
            "\n",
            "Iteration 4500:\tloss_train=167.385223:\tloss_val=170.503571:\tacc_train=0.770000:\tacc_val=0.759524\n",
            "\n",
            "\n",
            "Iteration 4600:\tloss_train=167.726196:\tloss_val=169.412888:\tacc_train=0.820000:\tacc_val=0.766667\n",
            "\n",
            "\n",
            "Iteration 4700:\tloss_train=168.787201:\tloss_val=168.334534:\tacc_train=0.720000:\tacc_val=0.778571\n",
            "\n",
            "\n",
            "Iteration 4800:\tloss_train=167.639130:\tloss_val=167.163986:\tacc_train=0.730000:\tacc_val=0.783333\n",
            "\n",
            "\n",
            "Iteration 4900:\tloss_train=168.792191:\tloss_val=166.183426:\tacc_train=0.720000:\tacc_val=0.771429\n",
            "\n",
            "\n",
            "Iteration 5000:\tloss_train=164.845474:\tloss_val=165.098129:\tacc_train=0.770000:\tacc_val=0.769048\n",
            "\n",
            "\n",
            "Iteration 5100:\tloss_train=162.636978:\tloss_val=164.088013:\tacc_train=0.750000:\tacc_val=0.785714\n",
            "\n",
            "\n",
            "Iteration 5200:\tloss_train=161.439240:\tloss_val=162.935333:\tacc_train=0.810000:\tacc_val=0.790476\n",
            "\n",
            "\n",
            "Iteration 5300:\tloss_train=156.675644:\tloss_val=162.016098:\tacc_train=0.800000:\tacc_val=0.776190\n",
            "\n",
            "\n",
            "Iteration 5400:\tloss_train=159.447449:\tloss_val=160.959900:\tacc_train=0.810000:\tacc_val=0.776190\n",
            "\n",
            "\n",
            "Iteration 5500:\tloss_train=160.192688:\tloss_val=159.966187:\tacc_train=0.810000:\tacc_val=0.785714\n",
            "\n",
            "\n",
            "Iteration 5600:\tloss_train=158.898544:\tloss_val=158.857986:\tacc_train=0.800000:\tacc_val=0.788095\n",
            "\n",
            "\n",
            "Iteration 5700:\tloss_train=160.601044:\tloss_val=157.967499:\tacc_train=0.780000:\tacc_val=0.785714\n",
            "\n",
            "\n",
            "Iteration 5800:\tloss_train=153.872284:\tloss_val=156.944962:\tacc_train=0.750000:\tacc_val=0.780952\n",
            "\n",
            "\n",
            "Iteration 5900:\tloss_train=160.072708:\tloss_val=155.980408:\tacc_train=0.710000:\tacc_val=0.795238\n",
            "\n",
            "\n",
            "Iteration 6000:\tloss_train=158.002213:\tloss_val=154.906296:\tacc_train=0.700000:\tacc_val=0.790476\n",
            "\n",
            "\n",
            "Iteration 6100:\tloss_train=150.956253:\tloss_val=154.042862:\tacc_train=0.780000:\tacc_val=0.790476\n",
            "\n",
            "\n",
            "Iteration 6200:\tloss_train=152.944427:\tloss_val=153.040848:\tacc_train=0.790000:\tacc_val=0.783333\n",
            "\n",
            "\n",
            "Iteration 6300:\tloss_train=149.106110:\tloss_val=152.112244:\tacc_train=0.780000:\tacc_val=0.797619\n",
            "\n",
            "\n",
            "Iteration 6400:\tloss_train=150.545959:\tloss_val=151.125290:\tacc_train=0.750000:\tacc_val=0.795238\n",
            "\n",
            "\n",
            "Iteration 6500:\tloss_train=148.487000:\tloss_val=150.239594:\tacc_train=0.760000:\tacc_val=0.792857\n",
            "\n",
            "\n",
            "Iteration 6600:\tloss_train=142.306320:\tloss_val=149.314407:\tacc_train=0.820000:\tacc_val=0.790476\n",
            "\n",
            "\n",
            "Iteration 6700:\tloss_train=141.618408:\tloss_val=148.416245:\tacc_train=0.880000:\tacc_val=0.800000\n",
            "\n",
            "\n",
            "Iteration 6800:\tloss_train=141.811783:\tloss_val=147.467194:\tacc_train=0.800000:\tacc_val=0.797619\n",
            "\n",
            "\n",
            "Iteration 6900:\tloss_train=142.965775:\tloss_val=146.568726:\tacc_train=0.850000:\tacc_val=0.800000\n",
            "\n",
            "\n",
            "Iteration 7000:\tloss_train=148.305313:\tloss_val=145.695328:\tacc_train=0.760000:\tacc_val=0.795238\n",
            "\n",
            "\n",
            "Iteration 7100:\tloss_train=140.375000:\tloss_val=144.830780:\tacc_train=0.820000:\tacc_val=0.797619\n",
            "\n",
            "\n",
            "Iteration 7200:\tloss_train=139.383728:\tloss_val=143.947296:\tacc_train=0.830000:\tacc_val=0.807143\n",
            "\n",
            "\n",
            "Iteration 7300:\tloss_train=144.177673:\tloss_val=143.014114:\tacc_train=0.800000:\tacc_val=0.804762\n",
            "\n",
            "\n",
            "Iteration 7400:\tloss_train=143.049347:\tloss_val=142.221222:\tacc_train=0.840000:\tacc_val=0.800000\n",
            "\n",
            "\n",
            "Iteration 7500:\tloss_train=148.032990:\tloss_val=141.370193:\tacc_train=0.750000:\tacc_val=0.800000\n",
            "\n",
            "\n",
            "Iteration 7600:\tloss_train=137.760345:\tloss_val=140.551147:\tacc_train=0.810000:\tacc_val=0.811905\n",
            "\n",
            "\n",
            "Iteration 7700:\tloss_train=138.816833:\tloss_val=139.620880:\tacc_train=0.800000:\tacc_val=0.814286\n",
            "\n",
            "\n",
            "Iteration 7800:\tloss_train=133.118225:\tloss_val=138.900208:\tacc_train=0.760000:\tacc_val=0.802381\n",
            "\n",
            "\n",
            "Iteration 7900:\tloss_train=134.157379:\tloss_val=138.060806:\tacc_train=0.750000:\tacc_val=0.804762\n",
            "\n",
            "\n",
            "Iteration 8000:\tloss_train=143.508728:\tloss_val=137.269745:\tacc_train=0.720000:\tacc_val=0.811905\n",
            "\n",
            "\n",
            "Iteration 8100:\tloss_train=134.096695:\tloss_val=136.375168:\tacc_train=0.800000:\tacc_val=0.814286\n",
            "\n",
            "\n",
            "Iteration 8200:\tloss_train=144.135132:\tloss_val=135.686630:\tacc_train=0.710000:\tacc_val=0.807143\n",
            "\n",
            "\n",
            "Iteration 8300:\tloss_train=132.954529:\tloss_val=134.869843:\tacc_train=0.780000:\tacc_val=0.804762\n",
            "\n",
            "\n",
            "Iteration 8400:\tloss_train=123.812744:\tloss_val=134.103516:\tacc_train=0.850000:\tacc_val=0.809524\n",
            "\n",
            "\n",
            "Iteration 8500:\tloss_train=138.412827:\tloss_val=133.236130:\tacc_train=0.780000:\tacc_val=0.816667\n",
            "\n",
            "\n",
            "Iteration 8600:\tloss_train=126.683357:\tloss_val=132.572220:\tacc_train=0.840000:\tacc_val=0.807143\n",
            "\n",
            "\n",
            "Iteration 8700:\tloss_train=119.054230:\tloss_val=131.775146:\tacc_train=0.900000:\tacc_val=0.809524\n",
            "\n",
            "\n",
            "Iteration 8800:\tloss_train=135.427612:\tloss_val=131.035751:\tacc_train=0.750000:\tacc_val=0.811905\n",
            "\n",
            "\n",
            "Iteration 8900:\tloss_train=126.261505:\tloss_val=130.227402:\tacc_train=0.760000:\tacc_val=0.816667\n",
            "\n",
            "\n",
            "Iteration 9000:\tloss_train=127.463264:\tloss_val=129.555969:\tacc_train=0.850000:\tacc_val=0.816667\n",
            "\n",
            "\n",
            "Iteration 9100:\tloss_train=138.103729:\tloss_val=128.813110:\tacc_train=0.670000:\tacc_val=0.811905\n",
            "\n",
            "\n",
            "Iteration 9200:\tloss_train=122.462776:\tloss_val=128.104782:\tacc_train=0.820000:\tacc_val=0.809524\n",
            "\n",
            "\n",
            "Iteration 9300:\tloss_train=125.869232:\tloss_val=127.337593:\tacc_train=0.780000:\tacc_val=0.814286\n",
            "\n",
            "\n",
            "Iteration 9400:\tloss_train=127.519249:\tloss_val=126.652344:\tacc_train=0.810000:\tacc_val=0.821429\n",
            "\n",
            "\n",
            "Iteration 9500:\tloss_train=123.441513:\tloss_val=125.954239:\tacc_train=0.820000:\tacc_val=0.814286\n",
            "\n",
            "\n",
            "Iteration 9600:\tloss_train=120.213623:\tloss_val=125.275002:\tacc_train=0.800000:\tacc_val=0.816667\n",
            "\n",
            "\n",
            "Iteration 9700:\tloss_train=122.089684:\tloss_val=124.549255:\tacc_train=0.780000:\tacc_val=0.821429\n",
            "\n",
            "\n",
            "Iteration 9800:\tloss_train=125.124695:\tloss_val=123.826599:\tacc_train=0.780000:\tacc_val=0.826190\n",
            "\n",
            "\n",
            "Iteration 9900:\tloss_train=117.591309:\tloss_val=123.196411:\tacc_train=0.800000:\tacc_val=0.819048\n",
            "\n",
            "\n",
            "Iteration 10000:\tloss_train=115.593460:\tloss_val=122.530533:\tacc_train=0.840000:\tacc_val=0.819048\n",
            "\n",
            "\n",
            "Iteration 10100:\tloss_train=123.534637:\tloss_val=121.863525:\tacc_train=0.760000:\tacc_val=0.828571\n",
            "\n",
            "\n",
            "Iteration 10200:\tloss_train=122.304779:\tloss_val=121.126144:\tacc_train=0.840000:\tacc_val=0.828571\n",
            "\n",
            "\n",
            "Iteration 10300:\tloss_train=122.379829:\tloss_val=120.569191:\tacc_train=0.770000:\tacc_val=0.821429\n",
            "\n",
            "\n",
            "Iteration 10400:\tloss_train=125.189018:\tloss_val=119.906433:\tacc_train=0.810000:\tacc_val=0.826190\n",
            "\n",
            "\n",
            "Iteration 10500:\tloss_train=116.488243:\tloss_val=119.282074:\tacc_train=0.860000:\tacc_val=0.828571\n",
            "\n",
            "\n",
            "Iteration 10600:\tloss_train=125.149551:\tloss_val=118.568489:\tacc_train=0.740000:\tacc_val=0.828571\n",
            "\n",
            "\n",
            "Iteration 10700:\tloss_train=117.868858:\tloss_val=118.047028:\tacc_train=0.810000:\tacc_val=0.828571\n",
            "\n",
            "\n",
            "Iteration 10800:\tloss_train=119.577744:\tloss_val=117.400452:\tacc_train=0.770000:\tacc_val=0.828571\n",
            "\n",
            "\n",
            "Iteration 10900:\tloss_train=110.638481:\tloss_val=116.789062:\tacc_train=0.850000:\tacc_val=0.830952\n",
            "\n",
            "\n",
            "Iteration 11000:\tloss_train=111.853516:\tloss_val=116.095467:\tacc_train=0.850000:\tacc_val=0.828571\n",
            "\n",
            "\n",
            "Iteration 11100:\tloss_train=122.342812:\tloss_val=115.590523:\tacc_train=0.790000:\tacc_val=0.828571\n",
            "\n",
            "\n",
            "Iteration 11200:\tloss_train=116.919266:\tloss_val=114.968124:\tacc_train=0.810000:\tacc_val=0.830952\n",
            "\n",
            "\n",
            "Iteration 11300:\tloss_train=108.811806:\tloss_val=114.381729:\tacc_train=0.840000:\tacc_val=0.828571\n",
            "\n",
            "\n",
            "Iteration 11400:\tloss_train=110.590828:\tloss_val=113.715347:\tacc_train=0.820000:\tacc_val=0.833333\n",
            "\n",
            "\n",
            "Iteration 11500:\tloss_train=113.746506:\tloss_val=113.214966:\tacc_train=0.760000:\tacc_val=0.830952\n",
            "\n",
            "\n",
            "Iteration 11600:\tloss_train=103.388443:\tloss_val=112.614708:\tacc_train=0.880000:\tacc_val=0.833333\n",
            "\n",
            "\n",
            "Iteration 11700:\tloss_train=111.711349:\tloss_val=112.062035:\tacc_train=0.820000:\tacc_val=0.830952\n",
            "\n",
            "\n",
            "Iteration 11800:\tloss_train=117.979210:\tloss_val=111.445610:\tacc_train=0.780000:\tacc_val=0.835714\n",
            "\n",
            "\n",
            "Iteration 11900:\tloss_train=103.607430:\tloss_val=110.919937:\tacc_train=0.830000:\tacc_val=0.838095\n",
            "\n",
            "\n",
            "Iteration 12000:\tloss_train=105.922920:\tloss_val=110.374786:\tacc_train=0.840000:\tacc_val=0.833333\n",
            "\n",
            "\n",
            "Iteration 12100:\tloss_train=113.601723:\tloss_val=109.838074:\tacc_train=0.770000:\tacc_val=0.830952\n",
            "\n",
            "\n",
            "Iteration 12200:\tloss_train=113.049370:\tloss_val=109.243294:\tacc_train=0.810000:\tacc_val=0.835714\n",
            "\n",
            "\n",
            "Iteration 12300:\tloss_train=108.944931:\tloss_val=108.700142:\tacc_train=0.810000:\tacc_val=0.840476\n",
            "\n",
            "\n",
            "Iteration 12400:\tloss_train=100.024925:\tloss_val=108.189095:\tacc_train=0.850000:\tacc_val=0.833333\n",
            "\n",
            "\n",
            "Iteration 12500:\tloss_train=108.231461:\tloss_val=107.676987:\tacc_train=0.860000:\tacc_val=0.833333\n",
            "\n",
            "\n",
            "Iteration 12600:\tloss_train=107.596970:\tloss_val=107.128174:\tacc_train=0.790000:\tacc_val=0.838095\n",
            "\n",
            "\n",
            "Iteration 12700:\tloss_train=111.895035:\tloss_val=106.549232:\tacc_train=0.780000:\tacc_val=0.842857\n",
            "\n",
            "\n",
            "Iteration 12800:\tloss_train=113.815605:\tloss_val=106.117676:\tacc_train=0.750000:\tacc_val=0.838095\n",
            "\n",
            "\n",
            "Iteration 12900:\tloss_train=110.494873:\tloss_val=105.601242:\tacc_train=0.780000:\tacc_val=0.838095\n",
            "\n",
            "\n",
            "Iteration 13000:\tloss_train=99.394722:\tloss_val=105.099518:\tacc_train=0.860000:\tacc_val=0.840476\n",
            "\n",
            "\n",
            "Iteration 13100:\tloss_train=101.473381:\tloss_val=104.528511:\tacc_train=0.840000:\tacc_val=0.840476\n",
            "\n",
            "\n",
            "Iteration 13200:\tloss_train=100.730423:\tloss_val=104.136665:\tacc_train=0.830000:\tacc_val=0.840476\n",
            "\n",
            "\n",
            "Iteration 13300:\tloss_train=102.612976:\tloss_val=103.629227:\tacc_train=0.850000:\tacc_val=0.838095\n",
            "\n",
            "\n",
            "Iteration 13400:\tloss_train=106.389656:\tloss_val=103.138123:\tacc_train=0.820000:\tacc_val=0.842857\n",
            "\n",
            "\n",
            "Iteration 13500:\tloss_train=103.076851:\tloss_val=102.584686:\tacc_train=0.820000:\tacc_val=0.842857\n",
            "\n",
            "\n",
            "Iteration 13600:\tloss_train=109.938522:\tloss_val=102.201675:\tacc_train=0.810000:\tacc_val=0.840476\n",
            "\n",
            "\n",
            "Iteration 13700:\tloss_train=93.335266:\tloss_val=101.709274:\tacc_train=0.860000:\tacc_val=0.842857\n",
            "\n",
            "\n",
            "Iteration 13800:\tloss_train=103.046394:\tloss_val=101.243576:\tacc_train=0.830000:\tacc_val=0.842857\n",
            "\n",
            "\n",
            "Iteration 13900:\tloss_train=105.301460:\tloss_val=100.696884:\tacc_train=0.770000:\tacc_val=0.845238\n",
            "\n",
            "\n",
            "Iteration 14000:\tloss_train=100.487335:\tloss_val=100.331680:\tacc_train=0.810000:\tacc_val=0.840476\n",
            "\n",
            "\n",
            "Iteration 14100:\tloss_train=94.755974:\tloss_val=99.844055:\tacc_train=0.890000:\tacc_val=0.840476\n",
            "\n",
            "\n",
            "Iteration 14200:\tloss_train=101.092567:\tloss_val=99.410439:\tacc_train=0.810000:\tacc_val=0.842857\n",
            "\n",
            "\n",
            "Iteration 14300:\tloss_train=96.309677:\tloss_val=98.905746:\tacc_train=0.820000:\tacc_val=0.847619\n",
            "\n",
            "\n",
            "Iteration 14400:\tloss_train=95.206993:\tloss_val=98.515137:\tacc_train=0.820000:\tacc_val=0.847619\n",
            "\n",
            "\n",
            "Iteration 14500:\tloss_train=94.505913:\tloss_val=98.081154:\tacc_train=0.850000:\tacc_val=0.840476\n",
            "\n",
            "\n",
            "Iteration 14600:\tloss_train=90.329330:\tloss_val=97.661682:\tacc_train=0.890000:\tacc_val=0.838095\n",
            "\n",
            "\n",
            "Iteration 14700:\tloss_train=92.418884:\tloss_val=97.172379:\tacc_train=0.870000:\tacc_val=0.847619\n",
            "\n",
            "\n",
            "Iteration 14800:\tloss_train=93.685867:\tloss_val=96.762260:\tacc_train=0.880000:\tacc_val=0.850000\n",
            "\n",
            "\n",
            "Iteration 14900:\tloss_train=98.393654:\tloss_val=96.352058:\tacc_train=0.790000:\tacc_val=0.842857\n",
            "\n",
            "\n",
            "Iteration 15000:\tloss_train=94.113258:\tloss_val=95.956825:\tacc_train=0.830000:\tacc_val=0.840476\n",
            "\n",
            "\n",
            "Iteration 15100:\tloss_train=89.607468:\tloss_val=95.501961:\tacc_train=0.860000:\tacc_val=0.847619\n",
            "\n",
            "\n",
            "Iteration 15200:\tloss_train=100.189362:\tloss_val=95.051018:\tacc_train=0.800000:\tacc_val=0.850000\n",
            "\n",
            "\n",
            "Iteration 15300:\tloss_train=88.643784:\tloss_val=94.703995:\tacc_train=0.840000:\tacc_val=0.845238\n",
            "\n",
            "\n",
            "Iteration 15400:\tloss_train=98.370605:\tloss_val=94.309990:\tacc_train=0.840000:\tacc_val=0.840476\n",
            "\n",
            "\n",
            "Iteration 15500:\tloss_train=92.680557:\tloss_val=93.896484:\tacc_train=0.840000:\tacc_val=0.852381\n",
            "\n",
            "\n",
            "Iteration 15600:\tloss_train=94.610123:\tloss_val=93.439140:\tacc_train=0.800000:\tacc_val=0.850000\n",
            "\n",
            "\n",
            "Iteration 15700:\tloss_train=90.530296:\tloss_val=93.137444:\tacc_train=0.790000:\tacc_val=0.847619\n",
            "\n",
            "\n",
            "Iteration 15800:\tloss_train=85.752975:\tloss_val=92.737511:\tacc_train=0.840000:\tacc_val=0.842857\n",
            "\n",
            "\n",
            "Iteration 15900:\tloss_train=95.323692:\tloss_val=92.344284:\tacc_train=0.800000:\tacc_val=0.852381\n",
            "\n",
            "\n",
            "Iteration 16000:\tloss_train=86.316261:\tloss_val=91.899422:\tacc_train=0.890000:\tacc_val=0.854762\n",
            "\n",
            "\n",
            "Iteration 16100:\tloss_train=93.606339:\tloss_val=91.614189:\tacc_train=0.810000:\tacc_val=0.847619\n",
            "\n",
            "\n",
            "Iteration 16200:\tloss_train=89.785629:\tloss_val=91.218422:\tacc_train=0.840000:\tacc_val=0.845238\n",
            "\n",
            "\n",
            "Iteration 16300:\tloss_train=74.686569:\tloss_val=90.843849:\tacc_train=0.870000:\tacc_val=0.854762\n",
            "\n",
            "\n",
            "Iteration 16400:\tloss_train=97.269302:\tloss_val=90.402466:\tacc_train=0.830000:\tacc_val=0.854762\n",
            "\n",
            "\n",
            "Iteration 16500:\tloss_train=83.147720:\tloss_val=90.129395:\tacc_train=0.890000:\tacc_val=0.850000\n",
            "\n",
            "\n",
            "Iteration 16600:\tloss_train=75.565964:\tloss_val=89.738968:\tacc_train=0.910000:\tacc_val=0.850000\n",
            "\n",
            "\n",
            "Iteration 16700:\tloss_train=96.209808:\tloss_val=89.390137:\tacc_train=0.780000:\tacc_val=0.852381\n",
            "\n",
            "\n",
            "Iteration 16800:\tloss_train=78.790298:\tloss_val=88.971573:\tacc_train=0.850000:\tacc_val=0.854762\n",
            "\n",
            "\n",
            "Iteration 16900:\tloss_train=87.468475:\tloss_val=88.685730:\tacc_train=0.860000:\tacc_val=0.850000\n",
            "\n",
            "\n",
            "Iteration 17000:\tloss_train=95.539330:\tloss_val=88.331375:\tacc_train=0.760000:\tacc_val=0.854762\n",
            "\n",
            "\n",
            "Iteration 17100:\tloss_train=79.930107:\tloss_val=88.001411:\tacc_train=0.870000:\tacc_val=0.854762\n",
            "\n",
            "\n",
            "Iteration 17200:\tloss_train=91.269119:\tloss_val=87.600311:\tacc_train=0.760000:\tacc_val=0.854762\n",
            "\n",
            "\n",
            "Iteration 17300:\tloss_train=87.103653:\tloss_val=87.294014:\tacc_train=0.840000:\tacc_val=0.854762\n",
            "\n",
            "\n",
            "Iteration 17400:\tloss_train=84.859940:\tloss_val=86.966370:\tacc_train=0.860000:\tacc_val=0.854762\n",
            "\n",
            "\n",
            "Iteration 17500:\tloss_train=84.003700:\tloss_val=86.653618:\tacc_train=0.810000:\tacc_val=0.854762\n",
            "\n",
            "\n",
            "Iteration 17600:\tloss_train=85.730286:\tloss_val=86.268112:\tacc_train=0.810000:\tacc_val=0.854762\n",
            "\n",
            "\n",
            "Iteration 17700:\tloss_train=80.595535:\tloss_val=85.924324:\tacc_train=0.850000:\tacc_val=0.859524\n",
            "\n",
            "\n",
            "Iteration 17800:\tloss_train=75.824615:\tloss_val=85.641342:\tacc_train=0.870000:\tacc_val=0.857143\n",
            "\n",
            "\n",
            "Iteration 17900:\tloss_train=76.898659:\tloss_val=85.333084:\tacc_train=0.870000:\tacc_val=0.857143\n",
            "\n",
            "\n",
            "Iteration 18000:\tloss_train=86.158546:\tloss_val=84.984901:\tacc_train=0.830000:\tacc_val=0.857143\n",
            "\n",
            "\n",
            "Iteration 18100:\tloss_train=82.780769:\tloss_val=84.617920:\tacc_train=0.880000:\tacc_val=0.861905\n",
            "\n",
            "\n",
            "Iteration 18200:\tloss_train=84.582748:\tloss_val=84.386711:\tacc_train=0.800000:\tacc_val=0.859524\n",
            "\n",
            "\n",
            "Iteration 18300:\tloss_train=89.575111:\tloss_val=84.066132:\tacc_train=0.860000:\tacc_val=0.859524\n",
            "\n",
            "\n",
            "Iteration 18400:\tloss_train=79.080002:\tloss_val=83.748344:\tacc_train=0.870000:\tacc_val=0.859524\n",
            "\n",
            "\n",
            "Iteration 18500:\tloss_train=89.247017:\tloss_val=83.392212:\tacc_train=0.820000:\tacc_val=0.864286\n",
            "\n",
            "\n",
            "Iteration 18600:\tloss_train=85.261482:\tloss_val=83.178368:\tacc_train=0.800000:\tacc_val=0.861905\n",
            "\n",
            "\n",
            "Iteration 18700:\tloss_train=80.706299:\tloss_val=82.859947:\tacc_train=0.870000:\tacc_val=0.859524\n",
            "\n",
            "\n",
            "Iteration 18800:\tloss_train=81.892494:\tloss_val=82.549606:\tacc_train=0.900000:\tacc_val=0.864286\n",
            "\n",
            "\n",
            "Iteration 18900:\tloss_train=73.309074:\tloss_val=82.194923:\tacc_train=0.880000:\tacc_val=0.866667\n",
            "\n",
            "\n",
            "Iteration 19000:\tloss_train=94.430771:\tloss_val=81.984665:\tacc_train=0.780000:\tacc_val=0.861905\n",
            "\n",
            "\n",
            "Iteration 19100:\tloss_train=91.149796:\tloss_val=81.675919:\tacc_train=0.800000:\tacc_val=0.861905\n",
            "\n",
            "\n",
            "Iteration 19200:\tloss_train=79.613365:\tloss_val=81.394638:\tacc_train=0.850000:\tacc_val=0.869048\n",
            "\n",
            "\n",
            "Iteration 19300:\tloss_train=78.346954:\tloss_val=81.040840:\tacc_train=0.870000:\tacc_val=0.866667\n",
            "\n",
            "\n",
            "Iteration 19400:\tloss_train=82.683006:\tloss_val=80.830711:\tacc_train=0.800000:\tacc_val=0.864286\n",
            "\n",
            "\n",
            "Iteration 19500:\tloss_train=77.305840:\tloss_val=80.535843:\tacc_train=0.870000:\tacc_val=0.861905\n",
            "\n",
            "\n",
            "Iteration 19600:\tloss_train=80.384590:\tloss_val=80.276932:\tacc_train=0.860000:\tacc_val=0.871429\n",
            "\n",
            "\n",
            "Iteration 19700:\tloss_train=91.245010:\tloss_val=79.946777:\tacc_train=0.810000:\tacc_val=0.869048\n",
            "\n",
            "\n",
            "Iteration 19800:\tloss_train=75.018539:\tloss_val=79.710335:\tacc_train=0.870000:\tacc_val=0.866667\n",
            "\n",
            "\n",
            "Iteration 19900:\tloss_train=70.764877:\tloss_val=79.449310:\tacc_train=0.890000:\tacc_val=0.864286\n",
            "\n",
            "\n",
            "Iteration 20000:\tloss_train=82.118874:\tloss_val=79.196823:\tacc_train=0.850000:\tacc_val=0.871429\n",
            "\n",
            "\n",
            "Iteration 20100:\tloss_train=79.408730:\tloss_val=78.869385:\tacc_train=0.830000:\tacc_val=0.873810\n",
            "\n",
            "\n",
            "Iteration 20200:\tloss_train=86.626038:\tloss_val=78.614304:\tacc_train=0.780000:\tacc_val=0.871429\n",
            "\n",
            "\n",
            "Iteration 20300:\tloss_train=70.698013:\tloss_val=78.374237:\tacc_train=0.860000:\tacc_val=0.871429\n",
            "\n",
            "\n",
            "Iteration 20400:\tloss_train=73.210754:\tloss_val=78.134583:\tacc_train=0.880000:\tacc_val=0.871429\n",
            "\n",
            "\n",
            "Iteration 20500:\tloss_train=76.868355:\tloss_val=77.834122:\tacc_train=0.790000:\tacc_val=0.873810\n",
            "\n",
            "\n",
            "Iteration 20600:\tloss_train=85.305481:\tloss_val=77.541054:\tacc_train=0.810000:\tacc_val=0.866667\n",
            "\n",
            "\n",
            "Iteration 20700:\tloss_train=83.203804:\tloss_val=77.357529:\tacc_train=0.830000:\tacc_val=0.873810\n",
            "\n",
            "\n",
            "Iteration 20800:\tloss_train=85.707031:\tloss_val=77.105400:\tacc_train=0.800000:\tacc_val=0.871429\n",
            "\n",
            "\n",
            "Iteration 20900:\tloss_train=69.773483:\tloss_val=76.836365:\tacc_train=0.900000:\tacc_val=0.873810\n",
            "\n",
            "\n",
            "Iteration 21000:\tloss_train=79.394241:\tloss_val=76.548470:\tacc_train=0.820000:\tacc_val=0.871429\n",
            "\n",
            "\n",
            "Iteration 21100:\tloss_train=75.725098:\tloss_val=76.384819:\tacc_train=0.830000:\tacc_val=0.873810\n",
            "\n",
            "\n",
            "Iteration 21200:\tloss_train=73.200111:\tloss_val=76.129402:\tacc_train=0.890000:\tacc_val=0.871429\n",
            "\n",
            "\n",
            "Iteration 21300:\tloss_train=79.279091:\tloss_val=75.870972:\tacc_train=0.830000:\tacc_val=0.876190\n",
            "\n",
            "\n",
            "Iteration 21400:\tloss_train=72.739059:\tloss_val=75.585342:\tacc_train=0.860000:\tacc_val=0.873810\n",
            "\n",
            "\n",
            "Iteration 21500:\tloss_train=79.177933:\tloss_val=75.423149:\tacc_train=0.870000:\tacc_val=0.876190\n",
            "\n",
            "\n",
            "Iteration 21600:\tloss_train=64.518539:\tloss_val=75.173935:\tacc_train=0.890000:\tacc_val=0.873810\n",
            "\n",
            "\n",
            "Iteration 21700:\tloss_train=77.080391:\tloss_val=74.939568:\tacc_train=0.810000:\tacc_val=0.873810\n",
            "\n",
            "\n",
            "Iteration 21800:\tloss_train=78.538513:\tloss_val=74.640915:\tacc_train=0.830000:\tacc_val=0.876190\n",
            "\n",
            "\n",
            "Iteration 21900:\tloss_train=71.687744:\tloss_val=74.492287:\tacc_train=0.850000:\tacc_val=0.873810\n",
            "\n",
            "\n",
            "Iteration 22000:\tloss_train=71.509689:\tloss_val=74.237503:\tacc_train=0.900000:\tacc_val=0.873810\n",
            "\n",
            "\n",
            "Iteration 22100:\tloss_train=70.502380:\tloss_val=74.032074:\tacc_train=0.860000:\tacc_val=0.876190\n",
            "\n",
            "\n",
            "Iteration 22200:\tloss_train=66.596100:\tloss_val=73.750069:\tacc_train=0.890000:\tacc_val=0.878571\n",
            "\n",
            "\n",
            "Iteration 22300:\tloss_train=70.336945:\tloss_val=73.577721:\tacc_train=0.870000:\tacc_val=0.873810\n",
            "\n",
            "\n",
            "Iteration 22400:\tloss_train=64.261047:\tloss_val=73.363792:\tacc_train=0.920000:\tacc_val=0.876190\n",
            "\n",
            "\n",
            "Iteration 22500:\tloss_train=74.408798:\tloss_val=73.161041:\tacc_train=0.870000:\tacc_val=0.876190\n",
            "\n",
            "\n",
            "Iteration 22600:\tloss_train=72.110535:\tloss_val=72.880135:\tacc_train=0.840000:\tacc_val=0.878571\n",
            "\n",
            "\n",
            "Iteration 22700:\tloss_train=65.878067:\tloss_val=72.687698:\tacc_train=0.890000:\tacc_val=0.876190\n",
            "\n",
            "\n",
            "Iteration 22800:\tloss_train=81.448257:\tloss_val=72.484100:\tacc_train=0.760000:\tacc_val=0.876190\n",
            "\n",
            "\n",
            "Iteration 22900:\tloss_train=70.581596:\tloss_val=72.297890:\tacc_train=0.870000:\tacc_val=0.876190\n",
            "\n",
            "\n",
            "Iteration 23000:\tloss_train=61.335590:\tloss_val=72.035843:\tacc_train=0.930000:\tacc_val=0.878571\n",
            "\n",
            "\n",
            "Iteration 23100:\tloss_train=77.555115:\tloss_val=71.804832:\tacc_train=0.840000:\tacc_val=0.876190\n",
            "\n",
            "\n",
            "Iteration 23200:\tloss_train=66.742912:\tloss_val=71.647408:\tacc_train=0.850000:\tacc_val=0.878571\n",
            "\n",
            "\n",
            "Iteration 23300:\tloss_train=76.583763:\tloss_val=71.454391:\tacc_train=0.850000:\tacc_val=0.878571\n",
            "\n",
            "\n",
            "Iteration 23400:\tloss_train=70.172958:\tloss_val=71.222504:\tacc_train=0.860000:\tacc_val=0.880952\n",
            "\n",
            "\n",
            "Iteration 23500:\tloss_train=71.798752:\tloss_val=70.984070:\tacc_train=0.850000:\tacc_val=0.880952\n",
            "\n",
            "\n",
            "Iteration 23600:\tloss_train=71.160423:\tloss_val=70.855919:\tacc_train=0.820000:\tacc_val=0.876190\n",
            "\n",
            "\n",
            "Iteration 23700:\tloss_train=72.473648:\tloss_val=70.650078:\tacc_train=0.830000:\tacc_val=0.876190\n",
            "\n",
            "\n",
            "Iteration 23800:\tloss_train=72.489159:\tloss_val=70.435005:\tacc_train=0.810000:\tacc_val=0.880952\n",
            "\n",
            "\n",
            "Iteration 23900:\tloss_train=60.801216:\tloss_val=70.200584:\tacc_train=0.890000:\tacc_val=0.883333\n",
            "\n",
            "\n",
            "Iteration 24000:\tloss_train=69.366867:\tloss_val=70.078056:\tacc_train=0.840000:\tacc_val=0.878571\n",
            "\n",
            "\n",
            "Iteration 24100:\tloss_train=74.823143:\tloss_val=69.866966:\tacc_train=0.850000:\tacc_val=0.880952\n",
            "\n",
            "\n",
            "Iteration 24200:\tloss_train=52.177116:\tloss_val=69.669739:\tacc_train=0.920000:\tacc_val=0.883333\n",
            "\n",
            "\n",
            "Iteration 24300:\tloss_train=63.374763:\tloss_val=69.424019:\tacc_train=0.910000:\tacc_val=0.883333\n",
            "\n",
            "\n",
            "Iteration 24400:\tloss_train=61.655220:\tloss_val=69.314079:\tacc_train=0.920000:\tacc_val=0.880952\n",
            "\n",
            "\n",
            "Iteration 24500:\tloss_train=56.435951:\tloss_val=69.102486:\tacc_train=0.940000:\tacc_val=0.883333\n",
            "\n",
            "\n",
            "Iteration 24600:\tloss_train=73.868011:\tloss_val=68.928726:\tacc_train=0.850000:\tacc_val=0.883333\n",
            "\n",
            "\n",
            "Iteration 24700:\tloss_train=56.027805:\tloss_val=68.684608:\tacc_train=0.910000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 24800:\tloss_train=74.894279:\tloss_val=68.562096:\tacc_train=0.850000:\tacc_val=0.883333\n",
            "\n",
            "\n",
            "Iteration 24900:\tloss_train=79.595146:\tloss_val=68.376709:\tacc_train=0.750000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 25000:\tloss_train=65.117416:\tloss_val=68.214760:\tacc_train=0.870000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 25100:\tloss_train=75.617142:\tloss_val=67.975906:\tacc_train=0.760000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 25200:\tloss_train=69.361206:\tloss_val=67.833420:\tacc_train=0.850000:\tacc_val=0.883333\n",
            "\n",
            "\n",
            "Iteration 25300:\tloss_train=68.011047:\tloss_val=67.662697:\tacc_train=0.870000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 25400:\tloss_train=64.582497:\tloss_val=67.511482:\tacc_train=0.840000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 25500:\tloss_train=66.132141:\tloss_val=67.278900:\tacc_train=0.830000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 25600:\tloss_train=63.290306:\tloss_val=67.101418:\tacc_train=0.860000:\tacc_val=0.883333\n",
            "\n",
            "\n",
            "Iteration 25700:\tloss_train=59.773273:\tloss_val=66.965660:\tacc_train=0.880000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 25800:\tloss_train=58.993137:\tloss_val=66.813820:\tacc_train=0.870000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 25900:\tloss_train=63.869610:\tloss_val=66.607620:\tacc_train=0.880000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 26000:\tloss_train=54.167030:\tloss_val=66.408264:\tacc_train=0.910000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 26100:\tloss_train=63.541168:\tloss_val=66.307724:\tacc_train=0.860000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 26200:\tloss_train=70.245712:\tloss_val=66.136635:\tacc_train=0.870000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 26300:\tloss_train=62.732391:\tloss_val=65.956169:\tacc_train=0.860000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 26400:\tloss_train=73.898277:\tloss_val=65.764297:\tacc_train=0.850000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 26500:\tloss_train=64.518837:\tloss_val=65.674141:\tacc_train=0.870000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 26600:\tloss_train=65.705994:\tloss_val=65.498283:\tacc_train=0.880000:\tacc_val=0.883333\n",
            "\n",
            "\n",
            "Iteration 26700:\tloss_train=65.560577:\tloss_val=65.323128:\tacc_train=0.910000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 26800:\tloss_train=59.304070:\tloss_val=65.121796:\tacc_train=0.890000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 26900:\tloss_train=72.320084:\tloss_val=65.034340:\tacc_train=0.840000:\tacc_val=0.883333\n",
            "\n",
            "\n",
            "Iteration 27000:\tloss_train=78.186646:\tloss_val=64.861862:\tacc_train=0.800000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 27100:\tloss_train=69.430016:\tloss_val=64.713699:\tacc_train=0.820000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 27200:\tloss_train=61.472633:\tloss_val=64.501625:\tacc_train=0.900000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 27300:\tloss_train=58.163860:\tloss_val=64.412621:\tacc_train=0.880000:\tacc_val=0.883333\n",
            "\n",
            "\n",
            "Iteration 27400:\tloss_train=63.707024:\tloss_val=64.247108:\tacc_train=0.860000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 27500:\tloss_train=64.893379:\tloss_val=64.119286:\tacc_train=0.860000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 27600:\tloss_train=70.412888:\tloss_val=63.917778:\tacc_train=0.860000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 27700:\tloss_train=60.150631:\tloss_val=63.806198:\tacc_train=0.890000:\tacc_val=0.883333\n",
            "\n",
            "\n",
            "Iteration 27800:\tloss_train=53.665691:\tloss_val=63.666866:\tacc_train=0.900000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 27900:\tloss_train=65.907372:\tloss_val=63.540169:\tacc_train=0.870000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 28000:\tloss_train=65.093529:\tloss_val=63.333988:\tacc_train=0.850000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 28100:\tloss_train=72.526855:\tloss_val=63.204208:\tacc_train=0.820000:\tacc_val=0.883333\n",
            "\n",
            "\n",
            "Iteration 28200:\tloss_train=51.931427:\tloss_val=63.077339:\tacc_train=0.910000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 28300:\tloss_train=60.789669:\tloss_val=62.957817:\tacc_train=0.860000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 28400:\tloss_train=61.388783:\tloss_val=62.770969:\tacc_train=0.840000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 28500:\tloss_train=64.711479:\tloss_val=62.607563:\tacc_train=0.880000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 28600:\tloss_train=66.800095:\tloss_val=62.525959:\tacc_train=0.840000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 28700:\tloss_train=76.400276:\tloss_val=62.388481:\tacc_train=0.810000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 28800:\tloss_train=59.689484:\tloss_val=62.225319:\tacc_train=0.870000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 28900:\tloss_train=59.266975:\tloss_val=62.067471:\tacc_train=0.870000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 29000:\tloss_train=61.453972:\tloss_val=61.996998:\tacc_train=0.860000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 29100:\tloss_train=61.090820:\tloss_val=61.852184:\tacc_train=0.870000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 29200:\tloss_train=63.335217:\tloss_val=61.697876:\tacc_train=0.870000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 29300:\tloss_train=55.869629:\tloss_val=61.534416:\tacc_train=0.880000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 29400:\tloss_train=62.632420:\tloss_val=61.463203:\tacc_train=0.890000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 29500:\tloss_train=53.331635:\tloss_val=61.318844:\tacc_train=0.880000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 29600:\tloss_train=66.677948:\tloss_val=61.189030:\tacc_train=0.830000:\tacc_val=0.890476\n",
            "\n",
            "\n",
            "Iteration 29700:\tloss_train=60.288574:\tloss_val=61.004333:\tacc_train=0.870000:\tacc_val=0.888095\n",
            "\n",
            "\n",
            "Iteration 29800:\tloss_train=59.544861:\tloss_val=60.947651:\tacc_train=0.860000:\tacc_val=0.885714\n",
            "\n",
            "\n",
            "Iteration 29900:\tloss_train=65.286079:\tloss_val=60.796352:\tacc_train=0.860000:\tacc_val=0.888095\n",
            "\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "s5rC6lSIAR-P",
        "colab_type": "code",
        "outputId": "ed9fa49d-40e6-4f4f-b965-74cb68adfa34",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        }
      },
      "cell_type": "code",
      "source": [
        "plt.plot(history['acc_train'],'b*')\n",
        "plt.plot(history['acc_val'],'r*')"
      ],
      "execution_count": 254,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7fd96c6ab240>]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 254
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztnXt8VOW1sJ89l1wmBIQk3ATEC+wI\nUalaRYZLotYi1jRUrBztxYqn+lWtespFqy3IZ0uBo/Z4qdVTrdVW8dKC6af0WBVomHha0YKCsq0K\nBEQlICQkk0zmsr8/dmayZzK3JJNkdljP74fOvr17rf1O1qz9vmutV9F1HUEQBMG62PpbAEEQBKFn\niCEXBEGwOGLIBUEQLI4YckEQBIsjhlwQBMHiOPr6hvX1R7sdJjN0qIvDh72ZFKffEF2yE9ElOxFd\noKSkUEl0zFIeucNh728RMobokp2ILtmJ6JIcSxlyQRAEoTNiyAVBECyOGHJBEASLI4ZcEATB4ogh\nFwRBsDhiyAVByBo8Hjsez8CJUOkr+jyOXBAEIRGrV+cA4Ha39LMk1kI8ckEQ+h2Px05VVT61tQ5q\nax1UVeWLZ94FxCMXBCGjhA2w2x1M+xq3O0hRkY+ZMw2TtHKlD1UN9Yp8AxEx5IIgZJTuDo9UVztY\nuNAX+bxoUVvGZRuoyNCKIAhdJt6kZHeHR8xtud1BFi9uE2+8i4hHLghCl4nndXd3eCTcFkBtrR23\nu4XKykCGJR7YiCEXhAzRnbFhq+Hx2Fm9OofaWsN0VFXls2hRW0TnrgyPxLYVpqoqn4svDlBWFurS\ns0z2/HvSN1boVzHkgpAhjoXQuVRed2lpKOJNV1cnNy+xbYVZudLHkiW5rF/ftWeZ7Pn3pG+s0K8y\nRi5EkGSM7tEXoXO90TddadN8btjrXrjQ18lYm4dE0hkeefBBJ2PGdPwQFBeH+MY3uvYskz3/VMfi\ntR3eb6WQSEXXu73OQ7foycISJSWF1NcfzaQ4/UY26lJVlQ/AunVd8zyyUZfu0l1ddu60MXNmAQA1\nNc0Zn6zrTt+k0qUrbZrPra52RHndPRnPrq52MHFiKOrZvf66naVL8yLbqhpKqUuy55/oWCL9zft7\no1+7+x1LtrCEGPJ+Ipt0iR2rnDYtEDXumYps0qWndFeXVas6JuwUhYyFzvWkbxLp0pU2e/rdSIfY\nZ2c2SeFnmapfkj3/2GPTpgXj6gR02j92rM7YsaG47aaD01MDgN89I7KvNwy5jJELx0QyRrwJq+5O\nYiW6zu0O4nYHU44Nd4Xu9E1Yvqqq9Ns8eFDB47FHdDLraD53/nx/wvt1d2Iy3rh6uuPsYZKNzYeP\nhYdLytnIaWe+ywO1uWxlCg/Mb+HE7dsACJ75JZy1uUxhKz88y8f7uVNwu4M4tr/DpoYv4fR06OjY\n/k5CeXbtMkatT/3779HRab57ZcfBqjlp6dQVxCPvJ7JNl554lNmmSzzivUbH25eOLum2lSm62jdh\nWTweR0JdYtsMG9aw/GZ9zOc++6yTsWNDPdb9rop/ALB0wzk4PTU4tr9DoOx0IL6BHDQoj6am1rTb\nj8djjzkBuKHgd/j/VQfAF4PHoSgKI327AXjfeyIAJ9l2gwKOU8aioGDbuycin9JwJLIvEa1NQXJC\nrTgwDH/I5UIfPoLg6OPJ8dSIRy50kMmwqK5EG2SS3g7tihcud/HFAdavd0T2VVS4mD/fz3XXdfY2\nu9pWbDheV2WFzs8i1qNMV77ycrjlFsPLjm3b3OaSJbl88IFxvKLCBcCOHWGvPp9p04w3jdWrczhp\n799gL1RVTefiiwN8tsaDc4edTZRHdC9nY0Qms1EOlJ3ORsp59c43uHbHjwB4+pxvczW/w/XFHoJj\nxyU2kIqCK47TGQxE2za7o/M5bT6FNj8sCAXIpwU7oYjhO/7gu1HnTmZb9MU7GiIfc2o3d5YrAa6Y\nbZvXC7t3Yd+9C8rLcd6yOGq4paeIR95PZEKX3vQCu0JPdOkLHeJNWJn3TZkSxOXSWbeupVuTapma\nEEv1LNJ5VmZZdv12I8cd58XvnkFVVT5fatjE3Xf7Ol1TV6fww5uNth+/6e/oOjzwYC4AP7zJx/Dh\nxp/s558rHH3wGQAG3/Rvxv7fPc2HH9q4hV/ywP0tnNiwjbw1T6OjdzLKwbHjCJx7HjlPP429tRkA\nHUjoZg5Ufvtb6i+5rMuXyWRnFtITXfpiAqordEeXvtQh3tDEqlU51NUpbNjgoL7eFpHhZz9zMHly\n+pNq06YFWbPGmXBCLNUbh8dj589/dvD3v9sjXvC0aYGohJhHHnGyZo0z6vjKi18F4GDZTACKt/+N\nKWzl3tfP5r3hs5ilb2DupsXk54d4gu+ya7edq3mC0OAhjBurM4TDKCjo6Bw4YDNmGBWFYY17CLX/\nDtkUQIHcXB0CAfQWH3bd0COIDZsCim6c3JIzhOb8IoYd3YMtlLgPs8VwtwwdSdNRKAl8BkC9cxQ2\nu05Rq7H9Rd4o8vJ1XIeNbf+UL9H2ySH2eEuY1PxW5JpCm/GDZLNDKAi+NoVQu4Vrw4mXApryijmt\n1bim5XvXkn/CGOp/8B9dllkMeRbSU116O9ytK2RryF6YeOFy4f9fdFE+W7c6IjJMn16QVJfYth5/\n3MnBgwqbN3s7HYf0vOzt2+2cdFIwSo4lS3IjHvQlqy/i1M83Meijd9nKFO6u+TLnLJnN7q2NrB/+\nHf5VMIVr9i5nClvZ2Tqe94ZO47KjT6F4271exUZIBzvZO4HtV0txajsBCI4YieL1Ro41NwOKQoFL\nNz4DBQUQCsHR5mjbVlig09LScU5TMwRDSuSYkuukMbeE/C+XcrQR/rXRMNQTKkZxtBHGsA+AV7UT\nyMvRmX6iMZbuP/9CONqId+cnbNjoYAz7mFAxild3juPQIYWiIsOsHaxXaGsf7vlfzmMqb3DakDrs\nduPHPFh6KgVf/hL15bO7/IzEkGchPdWlt8LdukOsLumOe3dFh65GnaTjCZvfCMaMCTF9eoDrrsvh\nyBFvStlTvVHE86Jj9bvzztzIcYCvD3mdE0/U+cMnFdTX29hAOcNsR/hh6JcsYxlfUrbS4ioCYGTz\nx4DhGQewk0vH+H62eL1gGOXgUS+BXFfEuwXDw7UdPAjBIKHCwejFxeAzhn1Cxx9PcPTx7P9EYcsW\nO/s/Nd6YcnN0fG2GZqNHhQgEFJqawNti7HPl6YRC0Np+jtOh4283qoWDdI4fHWL64rMiMlS/6OAd\n9TLq6hQm7fgjZWUh3i+7jPXrHYyuXQvA/mlzWbSojdGeP7Hf/Q2OPr4ucs2gv6zl8cYr4urtcukM\nH27IOHXfH3mByxk/Psh/nruGy9ZcJXHkYsgNMpmU0VNidUl33LsrOnQ1UqSr48k1Nc1omo2nnsrH\n7w+kNWaf7I2iqiofr5coL1tVQ1FyffrMZh6/WYtcc/cpj5Obq/P0num4m/7KKXwEQBAFOx1/NiF6\nLyX7gGMUzoDhDRcW6NhMN2poycGruGjzKxTQjA54KWDolLEM8tYTaPDy0eeFuGhm+Ph8nEWFhI4/\nnrVvnsAI3z5mFm9HOXqU4IknESouJjj6eLzLV5BTvRbHln/gXb4CgJzqtbRVzgWin/GTT3r5zneM\nacSammYWLMjD5dKjnvG3vpXHnj3Gj6OqBtE0e+SYptmivmPh71xVVT7//KcdpxPKyoL84he+Tv0a\n7rdrrvFH2nj4YWckcSmW5ctbGT1aj0p2KijQOeOMYNJoomSIIc9CBqIuvTHuHa/N2EgR8326IoP5\njWDvXoW9e22drgtHYIQjDMyhcj/zXMBJdZsYfWAbn444g8seOA/tEQ9vPbaDtbsNz2/W4Lexn3U6\noDNk1zvs2m0YlhPHB/n6kScpOroHux6IClULYCeAnTy69pbVyCAG09SxrZ5N864D2Nu86MChghMi\n47u+8vOx7/+kfUpSp9mrUHdkCO81jWMfYyIebPn5wYhn/KdP3UzlDQoH6ew8bioul85XCt+g7vhz\nqawMsPXht/h/Fav42obF1B1/LoXfq2LLkmp+9sF85vE86sQgZ6+spOLQCwARY52McB8VFOTy178G\nmDYtSF2dwhtvONi71/iVGTMmxIQJIT780BbZF2bMGGOeYdy4UKc3Io/H3umtCKCkJMT55wcZOzaU\n8HvhdgdZtSqHl192sG+fjcbGds+/UGfcuBBz5hjn3XhjLq+/7uDgwQ65zjgDli1L/dYXixjyLCST\nuvRGZbeutGnWJZ6X6vHY2b7d1uVqdmGeecbBze1RFYkiRQ4eVCLyJvOUw4Z4K1N4saGCO9yvRZI9\nBldO5ycz32QKW1lwv8rI+dMZUjUHpeEIvvlXESg7HdfqFTi2v0Nw7Dheufg/+Urt3Ti2v8PhwSeQ\ne92V5K15muBHdbzfOh6AsoJdhM44nUMHFUo+2UZTs4KDAAVKS2SiMF3acgrIaWuObB8++UxsDYcZ\ncnAXAA/xA64+4TXy2o6y234yg04ZiX/4CG59zpgQ/VX50xQWGtcGS08loJZGtf/OxMu4f+bLvMDl\nUR6sx2Onrk6J6gNNMwyTeb4h9g2rqEjntdfsPNgeAXP//S3Mnx/91pVsyCz8efFiI7OzstLPNdf4\ncbuDUd5wuI/jecjheyZ64zN/V8IsX97K9df7I+GelZWBuN+ncJhubImB2GdjPg6wYweUlIhHnklx\n+o1M6tKTEL506k2kwqxLvHHv8IReWVmwWzKeeabxRzB/vp+T925i/vwAP/NcEHUfj8fOlxo28Yv5\nb/LUu2cCMPrANhQFZv5wMn73DJyeGgbdeRu2vXvYyhRuKnuNDVREDPNvJt/DxX//OSM+3UpT/jCG\njhmEY4cRZ6zbbGC3o/gTj0X3xth0a9Vl5K37I2BEPIRTvuv22tjlmszU753Ch9UfRM5/p/QyZv/m\nUv5r+kvYHXa8cyojx07/4I/M/s2lQPTwRZhEcxZVVfnU1dkiWZ3pzslUVeWzZYs9Er6oKPDWW82d\nzoH4Q2ZhwmGhbncgsp0oEgmM74KiQF2dLe49Y3X2eOwRT37cOMPZCH9vwYhMivdczG2kOh7+cVIU\nuOgiBz/4gRjyTIrTb2RCl2TDCF2d7Js8Ocj8+X7KykJdGh7xeOwcd5yLI0eMcdVDh5SI57NqVQ7r\n1zuiXl3N94nXptNTw/bttkhY3at3vsH29oSTGya/xu0H/oPhw0NsmfxtTjstRG2t4TU1NNq4mic4\n2b6bxqHjGDECbHv34Pcr2E4aS+Dc88j7w1Movo7swDYc5NDhpcUa4oDiwKH37tzDJ4xisK0Z56Ac\nnIPz8Xrbx59dLhqDBRScWIx/2nTs2k5QDK/51tr5kck4dWIQX+VcFi9uI6d6LXfvvILaWnuk/yZO\nDLJypS9SOiDZPEQ8jzred8Hcx4k86ltvzWX37s7JS+ZJX/OwRrwhszCTJwdxOOxs29axXVYW5IEH\nOuqem98IVq3KobraEUlySvZ3ES/5bedOW8JnuGpVTpSusc813jM2lx2ornYweHA+5eViyDMpTr+R\nKV26Wtkt0bXmpJiuhAVWVeXjdDrw+zu8pUT3CLdnDq0LY/aYP96lcP+J93L33T4ci27jww9tbGY6\n/yfviY5EErsdcvNo8jmwhwLk6i1Jw+viectNuBiEN97pEXw4oyJCAEK5edhMPwghVwE2b4fX9y6T\nOY0dhl7h6AzgwAGFQYEjFBRAwObkg4YR7GAy7m+OoPCiL+PYYqStB84+J9JWvEnAVP2TybDOVG0l\n+p6ZwzrNmNuIDf2MHTIzXzN0aAFlZca2+bvaVbnTfdNMdX2Y7rxhSvVDMeSdiH2tGzxYjxv2lmiy\nr65OobbWmLAJn59OxbdEq7vE3m/Vqhy8LxupzZuUciYd2MSBehvLWMZo12FGDNfJHz2EtosvIe/u\nu3H4DIPYQh42BXJ1w2BmctjCb8vBGerQSXc6o4ZMWsgjnw5DfYQheMlnNEb43DNDr2es7yOme/8a\n2b7QsZFhjkbqfYNpdpVgDxo/UiXfuoD9O49GTRaOHhXisxPPpbg99vjd0ssir+/pzCGk8yoPUF+f\nS2trW8RzDRPrlcam/ZtluOkmY8x57NhQ1L0SvQ1CdAXBwkKdvDydCRNCTJsWZO9ehbKyUJTnHQ79\nvP9+X6fhkWnTgigKuFy5vP9+W9R3dfLkIHff7Us5kZ2s4mGi553q+jDpvLHGPlOpfih0IrZGyuOP\nO3E6O34rk1XLKy0NsXhxgJ07/RHvY+VKX1SYVqK6K+Vs5KQrFM6ovZhZbGQKW6MryW03zvtGnY2J\njb9n8BCdLZO/jbr/EXL4nEE0gxfYbfxz/v0NlGDHlz2fVkwRd2kb8S9GncqwT98HjMw7dD2SvQdG\ntMZHnMKkjY9G9rV+67vYdn1M7sbXAThc9W02rdvNbAxDfRdLubrqIJvXGWPRF075HO8JZ7PsiWlM\nYgfD9IP819Cl3Lz5Enb8tJqpvIF3+Qqqqx3M43mOWzSXMTttvDCzwJhIfK4Zr2ZjdvszbqsOdWkV\nmlR1ccLHL788l08/7Ty8EXsv85qZsTL88582iot1HnigLepeyaoymve//LKXP/zBwdlnGzJNn+5i\n715be4ifcc4zz7REJgjDsncejoAbbvBFfVedzsQ/fLHPqKtVJFNdHyZVO321upB45P1EpnWJl+CS\nKOwqlnQSc8x1lcNDIHsajuPX8//KVf/9VU5u2krj0OhKcgQCKD5fxEBnwqsOD1UoXi8tjX4CAYUQ\nRjp0c34JhScVY/P72L9f4b2mcSjofDlvByNcDThKT8LvnoF95/vYDtZHFD7AcP6nbjJ799mYxA7y\n82HvoFNpbVWY6N/BZyWT2Z1/Kr72ycHTP/gj6PD/8udRW+uIJHykG+qYjmeb6ZDNeN7y5MnGPWLD\n71KFeKbSKd7+RN/PWE8/EeG/lxtvzO30Bpnu8+ppEl2it4V03ljNcopHblF6o8JfbJtud5C6On/k\ni/PMMy2oaiitSoblbMTtDuJ3z6C62hEVK72RcgDmrF5BY4PCZ+dup2zNchRvMycDq/6rMDIsURhT\nSS6WRN9Cc2TGPzgbHTiXLUB0BmA4kaRt6jQCZ59D3bNbGDpU59nnnPwv53HN/DaGan/ns0W/oKhI\n54WZLwNw2o9bab6+ipzqtWg7bex1TWasW+fExV8np3othcC0TxQmLf02AM9f+gfUYbD77G/wOXDi\nlj/xFnNZvtj4g62uNgz6g5VhD/FyIPXbTzwvOpmn2N3vTbre8q9/3crbb9siYYVhwufPmhVM6sEm\n0ine/tjv5/e/38b11/s7XZuKiy4KcuON0W+Q6c4BmOUKR5Kkk8Eblj/R20I8+rrGv3jkfUB36153\ntU1zmF7YUzAb5bA3bd4GGNJe6L5h3ctRIXrBseP4VfPVzDnwJBO8RjnSILaoSUU9JxelrXNFvXT4\n3yFfYcrMAmyHDqK94+evyle5YNT2yHFVDYFCVAYgRCeSxNbKbmhQKCsLJgwZSxQKaU488Xp9aXtr\nmSiVkCzsD7o3obZqVQ4FBbk0N/uSesvPPGPU6A57xmYvM9NlIOJ9P9MlVYhrV0n32fa0D8KY5RSP\nPEtJllQTW7/a/BponmRKN2EmWU3sfftszGIjLeuD5F88HYDAnb+gYNc75Ew5jQb3jKiEFt/8q8hb\n83QkVnrYaROwffFFxMO27djO97mTekoi94+NDAkbcT0vD6XVmCBsGTqS3ICXQBBCDiOsDoBgkKNe\nBx/4xxPwtvE/DW6u/fhOFn26kKub7gPgjlFrqKwM8G7pZewq0gmuWYv9q3NxE4ybCVhaGqKoSOfO\nO3MjscC1tQ7+9S+FRx/tCBl75BEnjz3mjITE1dY6qKhwRSbLwt5WSUkujz2W2nMK91136rjHJkjF\ntpHqe5NKpnCbCxZAfX1bJ7nC7SxZkhsZonC5dCorAyxe3HksvKerHoX1Cd9r/XpHZFinO/Skdn66\nzzad81K9MfVljX/xyDNAsl/tRGFM5gQHoEsJM8lqYm+gnLNPPgRXX0Xumj/g3NHh4cZGZ+i5eYSK\nirDv/8TYttlQQum9/rW5Z5Lj+Vtku+V712J/ZxuaZuNA7limXj6C6hcd7Cicyu3nG1Er3uUrePWM\nn/B27lT+c7eRtn1zzRx0nYShXuk+l9iwtSlTgrzyijfSDhBV+yT2XmHS/Y71NAkrlV7dCSGMlSme\nLl1ZWDiTteIThbqmSyb/9tN9tr31fPot/FBV1fuAqRjzVTdrmvam6dgNwLeAILBF07RbkrU1kAx5\nbK2GdCeEPB47v/yli02bOrc5fnyQ++6LH1IV2+ZJdZuor1c4//wgh157l7IP/8zJ+4xJyTZnARsH\nX8JFh56LXNc8fBwFB+rS0i1YMhx7/YG4x9aNvo4543fQWneQtiHFFBfb2ecv4gqepbbWiNR4ZfC8\nSP2JeJNn5snY2MV2p00LdqqBkSzULPxMYuteJJrMi62HYaakpJB164wfgHjrVyaqapjOOHa82h6x\neoXvFf5/XZ2CohAVQmiWJ+zdx5uYrKpyRf5e4iWADR+uc9ZZxn17ewI2UahrTxeS7g6JQirjyRwm\nk8+nXwy5qqqzgEWapn1NVdVTgcc1TTuv/dhg4B3gFE3TAqqqvgL8VNO0/03U3kAy5BA/qcFMoqyv\nAwcKIwkOZsKeZLzVt8Pj25savsQ5i6cxpGoOH25pxOmAYcN0Cs6eGJk0BNBRaMbFh5xCGdsjRZli\n46Rj8ZWfj9LWxudHchn73muAMQnpwsvmEZfRFlBYcPkhVrw+nQ3F8/DcspaGxhbemXhZwkp1iVbS\nibfYbmVlIG4iUTKvNFHdC7O3b27LXA/DTGwqOER7XomqGqZLKr1iq+xVVeVH1TuPJw8Qt2Jf7N9L\nrFfs9ZKwjnqma8XH69eutJvJv/3p010UFxtvBMmyXZNlbPbk+fSXIV8O1Gma9pv27Z3AOZqmNaqq\nmodhyM8BmoCNwDWapn2QqL2BYsgT1bO+//70Jv4WLSqk+aXXqT9oYxPl2O06wWCHB/tiw/kM4XBk\n9e3t222Urvk5w/ZuIzRsGK2tNgZ99nFUmzoKzbZBDApFP6MmCoy47XbW8nXy8UbipMORISGXC724\nxPjxONoIgwqxazvRNBter8LvvJfzyBdXdPK4Z82CW27xRo3519baIxOOYW8mXU8IOtencLuDaSXN\nxHpR4a93qpCxsGf76qt5kTelWI/e5dLxeg2duxLemUqveOF5sfeO91YzeLAe6YN44Xyxfy833ZTH\ngQMK//qXLaVX3Fv17rvbbm+XtOgq/bFYeTJDjq7rSf9NnDjx0YkTJ37dtF0zceLEiabtqyZOnPjF\nxIkTP5k4ceI9qdrz+wP6QGH7dl03TIWu79ih6889l/61G5Zu0L3qGfoGZumg65vv3qDfzH36A9yg\nt6hndDQ8ZIiun3yy7rUVdOxL8a+eoqjtS3kx8vnQ/B/oa7hcX8pS/chF83R93jxdv/xyXb/1VkOw\n557rUKT9/889Z/zHrO+LL0brbjpd1/WO5sz7J03S9VmzOp8bj1gxnnvOuDZ8fbLrkrWR6N7htmP7\n1LxdVpZc53RIJJOuJ793+H6J+uDee+M/AzPh5x+v3XhypmqvO/RWu+mSju7p0E96JLSr6XjkjwIv\naZr2Yvv2Ztq97vahlTeAWUAj8Dpwg6Zp2xK1N1A8cuj6r3J4uATguKuvhAZjhe6Pxswg9EUDYwLG\nQrUtOYMY1vRJt+XynHEdIw+9R4G3nmaXEXHS5CrhQPGp6Ch8XnQqYKSGL1rUFrcSXjzM+po97kGD\ncpNWc+upJ9Sb63vGtn3CCTB1qj/i2e7ZEz2u211PPB0SvU2Et2NDAuO99ZhJVCe+q8k42UBvlbTo\nD937K/xwPzDStD0a+LT986nAx5qmHQRQVbUGOAtIaMj7mp4sB5ZOu4sjSSKOTsfMixKE47N1dPQh\nx0WMOMD4xm3YvY2R7fy2BnS7AyUYPXbXyCC8Yycwcu8/I/vCCwS0HGwhvzifeoZz1gmf47vpWmN5\nqS1/iqSMb6l2MH7Ln5i93Chl+s6qdlmL5oEneaVE6AinMuo02yKhaRs35iZ9VukmRyTqj9jrw+VU\n00noSEVs2zffDN/6ljF/UF3tQFWJSkBJlWjVk+9UvHC1RElEbneQn/40J+73L5485pT47ibjDAT6\nMiSwL0nHI58G3KVp2ldUVT0TuF/TtOntx0YAHuA0TdNaVFX9K7Bc07SaRO31tUfe0+XAetLuBioA\n8C66ncHf/TdsjYax9k/5Es6thjEOjhyJ7dChqLDAMMExY6G1laZDbQzRDcO/134Co0eF0F0udBTa\nvj6XgFoaqZIXL2kmlQ5hki0QbD4eGz6XTr+k4wkle6bh69escaIoRIp6ZSI0zixbvLeLrnhxmQzZ\n62n75onbVDW1s51sexvvCf3ikWuaVquq6luqqtZiLBd4g6qqVwMNmqatVVV1NbBBVdUAUJvMiPcl\nyQL6u5twkardcGha8Y4alrGMHIz4aeWqK7GZPG7voVZsJ5+Bq2E/9s9MC9Li4AhDKOEQAP8990X2\n/Y9G88H93MePALg3eDMFrlGdlswKG+50DHiiyoXxEpbM51VUuGhuplNSzYMPwuTJyVd7SeYJJUpy\nik2Qqq21R4Y4wsk/qfouHQ/ZLFu8t4t0vLiefKfSoavtGyGuRM4/eFCJqksuDCwGdEJQshChnoQP\nJbr2rop/0NoKj310Ad/ltzzBNQD8euxdXL93KWB44KHPDrJjiJtxf17FsJnnRtr96MYVvPWgsaZi\nRXkA15dP5e1L7+C1masj5+gofKVmYY/DwRLVfU5V2zo2nK+mppnp0wuorz/a5QWSk91nyZLcTtel\nK7OZrnrIPfH8Mh2y19P2zSGuvSFPXyIe+TFcjzzZK3FXXpdj6zWbt821in9WeyEA3x7zOm/Wn4g/\noPBasJzLeAFsdpT8PAY1dyTZHHIMxzFmBPmjBoOisJFyPi86lXfUy1AUuEN9lrt3XsHp2h95+OA3\nURS4vug5XnLNY/78QJfShWOPJ6vkZj433gRcbPjc6NG5/OY3waShcvEmKOPdJ3Zh3djFbhPJHE/f\n7kyQ9sRgxKthHdYvE4XTujrBcUVIAAAZZklEQVRR99BDhTQ3+9I+v6f0RnG4MDt2FHLkSNcXLM5G\npNZKF0n2StyVSY/Yes3hZA2np8aYcCNIuXcFx7UPpew+WIjdZyRafIvfG4k4IfBPOBm2dhjyI9ff\nSvGU0TRUziWnei32LXZmL7+U2Rj1Ltoq51JKiNmLL6Wt2hhDn115KXdPN9YYNNc4TlX3OPZ4skpu\n5nPjPafYa596irg10FNVz0t0n4cfDkUW0TVfl83V58zygfGszPploi51VyfqJk+G8vLkk6GZpDdr\nby9bBn5/Tq/X9bYqA9oj7y5mjzvZqiBzVn8VMCYz3/75a5z/5r0AHBp8AkWNezq165t9Cfu2G2Pl\nbX4b24tnUr5hUcL7xyvkk2qdw9j1CWPTt5OluqdTciD2/GQ10Lub4pzK80zX8+tOqFlvJJ6YE3cy\nGTqZir78e+mt8NCelETIVsQj7yPCnsW6dS1xVwV56qx7OOHWR7Dv3gWA47v/xqymFoLFJdgaGyNG\nvHnwSAoajcnM98q/z0k5B9j9tes4fXklD5b/hSNfhNqrfce/f6z34XYHu+T5htsxh54lW1UlUfuJ\niPV6Y0PzulODO9l1qZ5PLP0Vahar34MPtkTKFfTFm0Ff05tvP+vXO7r0nTxWEY/cRDzPIrx+pffl\nzSgK5F88neuf/QonfvE2tubmpO35Zl+CUn8Af8UFoCgE1FI2FM1L6L0k82ySlQRItb5gcbGR+FFf\nnzgtu7slBxLVvU5Fd7zl3vT8wvRG4kmqxJ3eoi8nCDOdaJMoUas3ErH6GvHIe5l4nkV4/cpA9V3k\neI/geAGG7O1YCSc0dBi2w190aqvle9eiHDpI43qj6FQ4e9JNYu8lmWcTz/uNXecQ4q8v+MMftlFe\nHky6qkqy9pORqO51b9Rq7o9x7+5i1i+dxB2rk+m3n9i+fvllKClpHbDPr6eIRx5DrGfx42mvMejO\n2yKLL+zOm8j41o6aYL7Zl0QWarDvrSPkKqCtsirigceL607mxWYq0iaddO9U1/SkoFFvJcb0doq1\nhLllD6kStayKeOR9QGlpiMuKNkTKxfrdM9CdHY/J2drERmZR4NIZf6LOsDf/TqiomIYX1+P66e0E\nzj6HtvYolFgjbk6OSbR6S6YibdJN9+5u+4no7cSYgZpiLXQmVaKW0IF45HEYUjUHx/Z3CA0bBgWF\nEW88zC5O4IXRN3HroN/g+GAnAG3TpuNddHtU/fBYzF6q1b0lM8nqXlstEWUg94uVEV3EI4+QbNw2\nvGiDeXk0W2MjujOn07n1DGfx/h+xbcqF/J4pADStvJegWhr3XrFhfea09r6mN5M2wlRXO1i40Bf5\nbPXJKUHIdo4pQ54sbM21egUARx9+LCptXvG3oefk8IkyhsH+QwwONXAOb7KBck4deTzNC28DILd6\nLd5Ft8e9V2xYn9MJ5eVQX59xFVPSm0kbYWT4QxD6ltRhCQMAj8dOVVU+tbVGYkxVVX7EM3V6ahha\n4SandjM5tZsZcuU8ApPLCJx8SuR6pa2NBl8eT4aujOx78+pf8krOJXgX/xjv4h8TaPfGY+9VUeGi\nosIVtejv1q12yss7p/73JsmeQaYxL4mVaBktQRAyxzFhyN3uIL/4RUc8dLieNcBGyjna6oxsfzb6\ndP5VVolv7jw+P+OCyP7LeZ6DDGdT+R00L7yNH5Q8FzWZaf58xRUd7V93XRsPP9x5fcxf/Sr58IZR\n8ztzhjbZMxAEwdocM5Od5lCmZ591MnZsiJcWvcLH313JmY1/A0DPySXgD/Fu4XmcfO8CXlj2IQBn\nu3bwefEkPi86NeWqOlVV+dTV2Zg/3x+pnR027OZiT6nCqXojfC/eM8hE+zIRlZ2ILtlJb0x2HhMe\nORjjtm53kNpao+BUba2DE757Md9u/FXkHKXNh1P3c2bj39jz/Xup22fj2n3L+SbPcdw0ldmPXRqJ\nwIgXWhgeuti3z8ajj+awb5+NvXttkSSdBQv8XHONn8WL25g0Kb6c8YZmMuWZx3sGvTnEIghC33DM\neORhwqFxs9jIj29vZf+KNQDkF+VxxaFHIudVr9jC128/C0g/hM4cdvfkk95IfY141yfTxdzOlClB\nXnnF20Ut05czE+GB4i1lJ6JLdiL1yDPQmeHhhat/dxHKkSMUBhv4KDSejZRzpf05AsOKyc2FDUoF\n2hU/BtLPIOzOwrjxuPHG3KgFfzNdUyTT2ZHyR5adiC7ZicSRZ4ByNvLV9Ytw1Hck+YxlDyXU8+mM\nbzDpuSXkVK9Ff9bR5foYmaqvcdFFwagFfzNdU0TCAwVhYHHMeeSPPOLkyuVncJL/g6j9a776KG+f\n/p1u177uKql06e2aIplEvKXsRHTJTsQj7yFOTw1XrLi9kxFv+d61XFq8i+Y4Xm9fJNDEQ7xmQRDS\n5ZiJWtEe8XDnnbkc9nZ4ultHXMi2id8gVFxCQC2NSl7pywSaeEhSjSAI6TLgXT2npwaAc9csYeru\nOhy0L7U2pAi1YgRPX/jfjOd52irnRg2jpFP7ui/qlvQnA10/QRgoDHhDXnDnEux767A1Nkbtv/8b\nr/Lv7neorAzQhhETHjuMkqr4U38Nu/QVA10/QRgoDFhDrj3ioXTNzyOVDMP4Zl9CoOw0zv/gBdoq\nlwDxa2hffLExnBEv8iTR+WVloQHhvfZ2TXFBEDLLgDXkS9ZfyInOUZEys38ccT3nH3qBwTvepfHJ\nZzi5ei1h/zreMMqSJdGF7M3j1InOX79+YHivVlpSTRCEATjZ6fHYqahwUVvr4JSta/lVyU94P/d0\npn2+lqGBg9j31qFUXIJeVBx13UMP5XDFFX6++c02rrwy9SRneNgl3fOtRli/hQt9EjUjCFnOgPsL\nNdf+3sFkLvzTHEbdtIGDW5si59h+fQ9+0yIQAP/8p43iYp1161p5+OEQS5fmAYm9UXN4YDrnWw0J\nfxQE6zCgPPJwyGCk9ndxEcO+MYfjtm7mFD7mi8JxbJn8LXKr13a65oMP7BGP+r337Cm9UfNQy9Gj\nyoDzXiX8URCsw8CwOu2Yx3ZnsZGf/tTH+4fvZdLSs40TXn6BbVoZE3g+7jVgeNSaZuuSNyreqyAI\n/cmAS9EPp7Zfv+Yr+HwKb+TOYm978akxY3WK71/SKfqiP9LhJeU4OxFdshPR5RhK0fd47JxUt4l/\n37EQxz6jKNYJ4w9wCQ+ziXLe+/enKI4TQicetSAIVmZAWS0jgeV8rnV2qPVn9Ud8ed55fBkfvz36\nTRbR2duW8WBBEKzMgDDk4QQWZ+1mfsktONkGGEu3Vb5xBxc8dQUg3rYgCAOTARG1El5YeBPltNGx\nkLLS5sPVeIAhVXNwemrE2xYEYUAyIAw5wPYHPXw4ZibnsAWA4IiRkWNNK+/F757RX6IJgiD0KgPG\nkOdcNIOhT/9nZLttztdovbSK5oW3RcWNC4IgDDTSGjRWVfU+YCqgAzdrmvam6dhY4BkgB3hb07Tr\ne0PQVBQV6Rx46EVyF94GgP2Dnfzlmj8AUHHohf4QSRAEoU9I6ZGrqjoLmKBp2nnAAuD+mFPuAe7R\nNO0cIKiq6rjMi5ma1atz+P0/T8O7+Md4F/8YX+VcVq/OYfXqHNoq5/aHSIIgCH1COkMrFwDrADRN\nex8YqqrqYABVVW3ADKC6/fgNmqbV9ZKscTGv5POzD+ZTVZXPI484mfP4lQOukJUgCEI8UmZ2qqr6\nKPCSpmkvtm/XAAs0TftAVdURQA3wF+BMoEbTtNuTtRcIBHWHI7NGdccOKCvr+DxpUvx9giAIFiaj\nmZ1KzOfjgf8CdgMvqap6iaZpLyW6+PBhbzduaZAotfWJJ3JYuDD82Uixj7cvm5CU4+xEdMlORBfj\nukSkY8j3AyNN26OBT9s/HwT2aJr2EYCqqq8Bk4GEhrw3CKfYOz01PPOMA49nuqTdC4JwzJDOGPkr\nwDwAVVXPBPZrmnYUQNO0APCxqqoT2s89C9B6Q9BkhA22a/UKzv3Lz1i9OkfS7gVBOGZIacg1TasF\n3lJVtRYjYuUGVVWvVlU1HApyC/Db9uMNwJ97TVoTHo89MoHp9NSgVFxCTu1mzmz8Gz+rvZC7Kv4h\nE5yCIBwTpDXmoGnabTG7tpmOfQhMz6RQ6WBe4d3vnkHu9/fAzTUA/IBf8eivT0BVZbFgQRAGPpbL\n7DSHG4ZDC7VHPAy6exlfFI7jlfPu4JfTnpFxcUEQjhksZ+1iV/T50Vmvce6K+di8DQwDypVN+KdN\nZ+8AWDdTEAQhHSxnyKFjhfeT6jYx6tEVOH0NkWOt86/CN/8qKpEJTkEQjg0sN7QCRrih2x3k9/vO\n51rfQ5H9H8/+d+x7+zSxVBAEod+xpEceDicsKvLx2szneY55TP3eBEYUQ0At7WfpBEEQ+hZLGvIw\n1dUOTr90Im+qt7NFgTvUZ6VAliAIxxyWNuSlpSFmL76U2bRRXe0QIy4IwjGJJcfIw4SHWDweO0VF\nyYt/CYIgDFQs7ZGHMScHCYIgHGtY2iP3eOzcVfEPnLWbpe64IAjHLJb2yN3uIOXOZWzFTgUbWbnS\nhyqJQIIgHGNY1iN3emoYUjWH47ZuppxNfDhmJtsf9PS3WIIgCH2OZT1yv3sGTUXFDJt5LgBDn7kH\np1YGktEpCMIxhmUNOUBu9VqaF94W+Vy5SJKBBEE49rCsIXd6jJK13sU/BiCnem1/iiMIgtBvWNaQ\nu1aviNqWZCBBEI5VLDfZGZ7kzKndTE7tZoZUzYl454IgCMcilvPIYyc5m1beS1AKZQmCcAxjOY8c\nOiY5mxfeRq6MjQuCcIxjOY88jN89A797hkxyCoJwzGNJj9xZuzky2SmTnIIgHOtYy5Bv3CgTnYIg\nCDFYy5CXl9P0i3sim00r78XvntGPAgmCIPQ/1jLkyESnIAhCLJab7AyUnhoZF5eJTkEQBAt65ObJ\nTZnoFARBsKAhFwRBEKIRQy4IgmBxLGfInZ4aCTkUBEEwYbnJznAiUIOEHQqCIAAW8sidnhooL48k\nAykVl4hnLgiCgIUMud89Ax56KLJ9s/NBSQYSBEHAQoYcYPfq53lszE9YxlJO2bqWqqp8PB57f4sl\nCILQr1hqjHz8JZOZuGAO184sYB7Ps3KlD1UN9bdYgiAI/YqlPHIuv5zqagcLF/oYv7CS6mpL/Q4J\ngiD0CpazhKWlISorAwBiyAVBELCaRw4RIx77WRAE4VglLZdWVdX7gKmADtysadqbcc5ZAZynaVp5\nRiUUBEEQkpLSI1dVdRYwQdO084AFwP1xzpkEzMy8eIIgCEIq0hlauQBYB6Bp2vvAUFVVB8eccw9w\nR4ZlEwRBENIgnaGVkcBbpu369n2NAKqqXg1sAnanc8OhQ104HN2P/S4pKez2tdmG6JKdiC7ZieiS\nmO6EfSjhD6qqDgO+B1wIHJ/OxYcPe7txS4OSkkLq6492+/psQnTJTkSX7ER0SW780xla2Y/hgYcZ\nDXza/vl8oASoAdYCZ7ZPjAqCIAh9RDqG/BVgHoCqqmcC+zVNOwqgadoLmqZN0jRtKjAXeFvTtFt7\nTVpBEAShEykNuaZptcBbqqrWYkSs3KCq6tWqqvbLOmsej13qqwiCIJhIa4xc07TbYnZti3PObqC8\n5yIlZ/XqHADc7pbevpUgCIIlsExmp8djp7wcamsd1NY6pPKhIAhCO5YpVuJ2B5kwAcrKjG2pfCgI\ngmBgGY8c4PnnYeFCHwsX+qRgliAIQjuWsoaTJ0N5eRsglQ8FQRDCWMojv/zyjs9S+VAQBMHAUoZc\nEARB6IwYckEQBIsjhlwQBMHiiCEXBEGwOGLIBUEQLI4YckEQBIsjhlwQBMHiiCEXBEGwOGLIBUEQ\nLI4YckEQBIsjhlwQBMHiiCEXBEGwOGLIBUEQLI4YckEQBIsjhlwQBMHiWMqQb9yIrNMpCIIQg6WW\n2Vm2DPz+HNzulv4WRRAEIWuwhEfu8dipqspn0yaorXVQVZUvnrkgCEI7lvDI3e4gRUU+Zs40xF25\n0oeqhvpZKkEQhOzAEh45GIstL10KCxf6ZOFlQRAEE5axiKWlIRYsgPr6NjHkgiAIJizjkVdWBuJ+\nFgRBONaxjCEHYONGnJ6a/pZCEAQhq7DWGMWyZbj8QRrcM/pbEkEQhKzBEh6501PDkKo5sGkTObWb\nGVI1RzxzQRCEdizhkfvdM2gqKmbYzHMBaFp5L0G1tJ+lEgRByA4s4ZED5FavhaVLaV54m/FZEARB\nACzikQMESk+FBd/BW3+UHDHkgiAIESzjkbdVzo37WRAE4VjHMoZcEARBiI8YckEQBIuT1hi5qqr3\nAVMBHbhZ07Q3TccqgBVAENCAazVNk4pWgiAIfURKj1xV1VnABE3TzgMWAPfHnPIoME/TNDdQCMzO\nuJSCIAhCQtIZWrkAWAegadr7wFBVVQebjp+ladq+9s/1QFFmRRQEQRCSkc7QykjgLdN2ffu+RgBN\n0xoBVFUdBVwE/CRZY0OHunA4ur8oRElJYbevzTZEl+xEdMlORJfEdCeOXIndoarqcODPwA80TTuU\n7OLDh73duKVBSUkh9fVHu319NiG6ZCeiS3YiuiQ3/ukY8v0YHniY0cCn4Y32YZb1wB2apr3SZekE\nQRCEHpHOGPkrwDwAVVXPBPZrmmb+ObkHuE/TtL/0gnyCIAhCClJ65Jqm1aqq+paqqrVACLhBVdWr\ngQbgf4DvABNUVb22/ZKnNU17tLcEFgRBEKJJa4xc07TbYnZtM33OzZw4giAIQleRzE5BEASLI4Zc\nEATB4oghFwRBsDhiyAVBECyOGHJBEASLI4ZcEATB4oghFwRBsDhiyAVBECyOGHJBEASLI4ZcEATB\n4oghFwRBsDjdqUfeLzg9NXCcCyaf1d+iCIIgZBWWMeSu1SvAaYfn/9zfogiCIGQVWT+04vTUMKRq\nDjm1m2HTJoZUzTG8c0EQBAGwgEfud8+gqaiYYTPPBaBp5b0E1dJ+lkoQBCF7yHqPHCC3ei3NC2+D\npUvJrV7b3+IIgiBkFVnvkQMESk+lrXIuBSWFBB57sr/FEQRByCos4ZG3Vc6N+1kQBEGwiCEXBEEQ\nEiOGXBAEweKIIRcEQbA4lpjsBPB47Bx3HEye3N+SCIIgZBeWMeSrV+fgdMLzz/e3JIIgCNlF1g+t\neDx2qqryqa11sGkTVFXl4/HY+1ssQRCErCHrPXK3O0hRkY+ZMw1RV670oaqhfpZKEAQhe8h6jxyg\nutrBwoU+li41PguCIAgdWMIqlpaGqKwMUFKSy2OPiTcuCIJgxhIeeWVlIO5nQRAEwSKGXBAEQUiM\nGHJBEASLI4ZcEATB4oghFwRBsDhiyAVBECyOGHJBEASLo+i63t8yCIIgCD1APHJBEASLI4ZcEATB\n4oghFwRBsDhiyAVBECyOGHJBEASLI4ZcEATB4oghFwRBsDiWqEcOoKrqfcBUQAdu1jTtzX4WKW1U\nVS0Hngd2tO96F1gFPAXYgU+Bb2ua5usXAdNEVdUy4EXgPk3THlRVdSxxdFBV9SrgFiAEPKpp2mP9\nJnQc4ujxBHAWcKj9lNWapr2U7XoAqKq6CpiB8be8AngTC/YJxNWlEgv2i6qqLuAJYASQB/xfYBu9\n2C+W8MhVVZ0FTNA07TxgAXB/P4vUHTZpmlbe/u8mYDnwkKZpM4APgWv6V7zkqKpaADwAvGba3UmH\n9vN+ClwIlAO3qqo6rI/FTUgCPQBuN/XPS9muB4CqqhVAWfvfxWzgl1iwTyChLmDBfgEuBbZomjYL\n+CZwL73cL5Yw5MAFwDoATdPeB4aqqjq4f0XqMeVAdfvnP2N0ZjbjA+YA+037yumsw7nAm5qmNWia\n1gJ4AHcfypmKeHrEI9v1APgbcHn75yNAAdbsE4ivS7xV1rNeF03TntU0bVX75lhgH73cL1YZWhkJ\nvGXarm/f19g/4nSLSaqqVgPDgLuAAtNQygFgVL9JlgaapgWAgKqq5t3xdBiJ0T/E7M8KEugBcKOq\nqv+BIe+NZLkeAJqmBYHm9s0FwMvAV63WJ5BQlyAW7JcwqqrWAmOArwGv9ma/WMUjj0XpbwG6yL8w\njPfXge8CjxH9I2o1feKRSAcr6PYUcJumaecDW4Flcc7JWj1UVf06hvG7MeaQ5fokRhdL94umadMw\nxvl/T7ScGe8Xqxjy/Ri/XmFGY0wYWAJN0z5pf93SNU37CPgMY3gov/2U40n9qp+NNMXRIbavsl43\nTdNe0zRta/tmNXAaFtFDVdWvAncAF2ua1oCF+yRWF6v2i6qqZ7UHAtAuvwM42pv9YhVD/gowD0BV\n1TOB/ZqmHe1fkdJHVdWrVFVd2P55JMZs9m+By9pPuQz4Sz+J1xNepbMOfwe+rKrqcaqqDsIY86vp\nJ/nSQlXVP6qqelL7ZjmwHQvooarqEGA18DVN075o323JPomni1X7BZgJ/AhAVdURwCB6uV8sU8ZW\nVdVfYDygEHCDpmnb+lmktFFVtRB4GjgOyMEYZvkn8CRGeNIe4Huapvn7TcgUqKp6FnAPMB7wA58A\nV2GEWUXpoKrqPGARRqjoA5qm/aE/ZI5HAj0eAG4DvEAThh4HslkPAFVVv48x3PCBafd3gd9goT6B\nhLr8FmOIxWr9ko8xfDoWyMf4e99CnL/3TOliGUMuCIIgxMcqQyuCIAhCAsSQC4IgWBwx5IIgCBZH\nDLkgCILFEUMuCIJgccSQC4IgWBwx5IIgCBbn/wMWjJUtGHXOXAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "tkToA3CjBxJe",
        "colab_type": "code",
        "outputId": "c1dd28a6-ea31-44e4-e41d-1225b00cd7f0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        }
      },
      "cell_type": "code",
      "source": [
        "plt.plot(history['loss_train'],'b*')\n",
        "plt.plot(history['loss_val'],'r*')"
      ],
      "execution_count": 255,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7fd96c6ab0f0>]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 255
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAD4CAYAAAATpHZ6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztnXl4lOXV/z+zJCEJASFEQAmCInc0\ngAhiJaMQrFZciEFFqdRq5VdtBau2iPbVClr6ilCXKmj1FaS1rVawQNTa1g2XiVVcUAlyo8gSQCAE\nTCDLZLbfH8/MZDKZSSYbeSY5n+viYuaZZ7nPPJPvnDn3uc+x+P1+BEEQhK6BtbMHIAiCILQfIuqC\nIAhdCBF1QRCELoSIuiAIQhdCRF0QBKELYe/Mi5eVHW516k2fPmkcOlTdnsPpNMQWcyK2mBOxBbKy\nMiyxXktYT91ut3X2ENoNscWciC3mRGxpmoQVdUEQBKExIuqCIAhdCBF1QRCELoSIuiAIQhdCRF0Q\nBKEL0akpja1FP+lkd88eHD9jbGcPRRAEwVQkpKinLb4fLBaY8VJnD0UQBMFUJFT4RT/ppHTYJYyp\nfIfRFW9Tnn02+klnZw9LEATBNCSUqKsbHfgf+33ouceajLrR0YkjEgRBMBcJJeoAh+95lFLbCQCM\nqPlIvHVBEIQwEk7UvRdOZv/ylaHnSb7aThyNIAiCuYhrolQptQg4J7D//cB64BkgCXADP9Ja71VK\nuYFwt/n7Wmtvew541H0FlIy7iV22Exjk3cFJ7i1k/mY6G3leQjGCIHR7mhV1pdQkYITWerxSKhP4\nFHgLeEpr/YJSahbwS2AuUKG1zu/IAYPhrW+rO5dBy68H4BgqOPb+O9A8IMIuCEK3Jp7wyzvAtMDj\n74B04CbgxcC2MiCz/YcWm1H3FeDduoM31I2hbb37JYmgC4LQ7bH4/fGXNFdK3QCco7W+JvDcBrwJ\n3Ke1fkMpdQQoAk4AXtRaP9TU+Twer7+1pSff/+VKUt98hSHb13FMxQ4Ajpx0Gl/PfoTRt+a36pyC\nIAgJQsx66nEvPlJKXQrMBH4QeG4DngXe1Fq/EdhtDvAXwA+8o5R6R2v9UaxztqXQ/fiHpvFsgZvf\nW27jLxtGA7B7aw1/+IOXRTMOt/q8nUFWVgZlZYk15liILeZEbDEnrbUlKysj5mtxZb8opS4A7gIu\n1FpXBDY/A3yltb43uJ/W+o9a6yNa6yrgDWBki0fbAiYvm8Lv+y9iO0aKo2ILD2+/nHsnfYjT2XUK\n6QuCIMRLPBOlvYHFwHla64OBbTOAOq31vLD9FDAPmAHYAAewqiMGHc5zlZdQ2uf7PHRoJmBMmt65\n7zZ6shA353T05QVBEExFPOGXq4B+wAuGbgMwGPhOKbUu8HyT1vompVQp8CHgA4q01h+283gb0Wtm\nIbe891uWrvg5s3gCgAMVKaQ4RNAFQeh+NCvqWuungKfiOZnW+o42j6iFFBR4+NfaEQzp9y9KD51A\ntncHo90fsWXI2ez/taQ4CoLQvUi4FaXRqLt0KmcvPK/BStNeybLSVBCE7keXEPWCAg91BVPxPfAo\nB3sNBmDAd1s4bf50kpzvdvLoBEEQjh5dQtSDbD7pQh7seU/oeYa3gr3Tfy0FvwRB6DZ0KVGfvGwK\nN/xgK0v5eWjbsb1rGTHC14mjEgRBOHp0KVEHeKd8BEP6HeHbZCN3ve/+LaTPuBr9pFNy1wVB6PJ0\nOVGvu3QqB8dPZnn23aFtSdUV9FtwBztue6ITRyYIgtDxdDlRLyjwMHnZFM47eVuDMEylK4XTtr9E\nYWGqeOyCIHRZupyoB9mSlMvxp2WygmvZTz/OZD35vM2/951OPus6e3iCIAgdQpcV9bpLp3J48CmU\nzHmSn5zwRmj7wT11bNzYZc0WBKGb02XVraDAQ92lU5k7t46Hj1vE4Uwjf31wjeSvC4LQdemyog6G\nsAN855jMwhTJXxcEoevTpUU9yNC5l0r+uiAI3YJuIeoQPX+917U/lDCMIAhdim4j6tHy162VlVhu\nu1OEXRCELkO3EfVg/vqPzvmmQRjGWifVHAVB6Dp0G1EP8k75CM489TClNiMMk7HnK9JnXC3euiAI\nXYJuJ+qfq8t5rvISfuMNdeIjqbpCwjCCIHQJup2oz51bx9S/XcIQdjQIw5TvdsmiJEEQEp5uqWJF\nRXbcw3PI7nOE7RhhmBPdW8hbfJV464IgJDTxNJ5GKbUIOCew//3AeuBZwAZ8C1yjtXYppWYAt2I0\nnn5Ka72sQ0bdRnJyfJBzKVuX+fnP5/ksqZoJhGXDPLwQtzSuFgQhAWnWU1dKTQJGaK3HA5OBR4D7\ngKVa63OAr4HrlVLpwD3AeUA+cJtSqm9HDbwtFBR4KCjw0GtmIbdP+7pBGCbdJtkwgiAkLvGEX94B\npgUefwekY4h2UWDbSxhC/j1gvda6QmtdAzgBR7uOtp0pKPA0yoaxb/2KHtOvlhICgiAkJM2GX7TW\nXqAq8HQm8E/gAq21K7BtPzAQGACUhR0a3B6TPn3SsNtbX9s8Kyuj1ccG2Trmar7emsQO77ms4HoA\nergqGLDwTrLOfgTy89t8jXhoD1vMgthiTsQWc9LetsQVUwdQSl2KIeo/AL4Ke8kS45BY20McOlQd\n7+UbkZWVQVnZ4VYfH2TWLHiu7yVYbnmApfycWRjdkdxVtVx2mZcxv6zlxhvdbb5OU7SXLWZAbDEn\nYos5aa0tTX0RxJX9opS6ALgLuFBrXQEcUUqlBl4+HtgT+Dcg7LDgdtNTWmolK1+RTnUoG+YUNMsP\nXc7e5yUMIwhC4hDPRGlvYDFwidb6YGDz68DlgceXA/8CPgDGKaWOUUr1xIinJ0R+YE6Oj+1nXMZX\nwxuW6D2GCv5fya+4eeSHPPlkUieOUBAEIT7i8dSvAvoBLyil1iml1gG/A65VSr0L9AX+FJgcvRP4\nN4bo3xvw6k1PQYGHnBwfJwZK9K4fd0PotRRqsdng1VfjjlQJgiB0Gha/399pFy8rO9zqi3dUXC25\naDVbHnqNQV+/y8C6HQB8R28KWcOB3HNYsMCFw+Ft12tKjNCciC3mRGyBrKyMmHOW3XJFaVPUFUyl\ncsJk3pl0V2jbMVTwCLdy2qF17S7ogiAI7YmIehRG3VfAJSO/Yc+l/y+0LYVadu+xUViYitPZ+jRM\nQRCEjkREPQaenFPY/VUt+1KNhtWnoHmZS3h27IPirQuCYFpE1GMQDMOkLrwztK0nVQx86n4p+iUI\ngmkRUW+CUfcVYCvdyfbTLg5tS3VVkH73HSLsgiCYEsnTawZPzilkAdWnDyBthVF00lLravogQRCE\nTkI89WaoK5hKCbm89XJdaLWpfau0wBMEwZyIqMfB0LmXcuLN5zMfaYEnCIK5EVGPk+WHr+Tq8Vtx\nnla/2vTQXgnDCIJgLkTU4yQnx8cxjhy+/bomFIYZUruFpMuuZO+dT3Ty6ARBEAxE1OOkoMDD0LmX\n8kHWRQ3CMOn+KnJX/U7CMIIgmAIR9ThxOo3VpL/fPp0h7OBl65TQa6HepiLsgiB0MiLqceJweFm4\n0Iihl5DLqGtH8NdeN4ZeL9/tYuNGeTsFQehcRIVaQFGRnTlzXFivvJT7i07DXVkbiq+f6N7C6N9c\nwd47n8DptEl9GEEQOgVZfNQCcnJ8FBR4AHjiiUt5ZZ6VdUwM9TbtSRVD/vS/PPTmWLYcNxGHo6Yz\nhysIQjdEPPUWEBR0gMOHLQyZU8A5g7bxZq/6+HqGt4Jbtv+KpOL3pKKjIAhHHfHUW0nQa//8iGLf\nm36cQwfi+OwpwCjTC/DAAy6U8oWEXao7CoLQ0Yin3kqCXvvhC6biUaeQ3beKb5Prm1a/zCVsvdmI\nr999dwqLFyd35nAFQegmxOWpK6VGAGuBh7XWS5RSK4GswMt9gf8C/wt8AXwc2F6mtZ7WzuM1HYZY\nX0nB9R4+HHAOA5/7GWDE16/8Yj7zrrRR4r4NgMLCVG6/vU48dkEQOoxmRV0plQ48BrwR3BYu1kqp\n5cDT9S/p/HYeoylxOm0sXpxMcbHxFl7E1RRu+C3elClMdr0EQKq3iv/x3suHnM7b5DN9ulsEXRCE\nDiWe8IsLuAjYE/mCUkoBx2itP2zvgZmd8Lx1gOpqCx9Wj+C/rjG80bN+4jTY3/T2ca9TWirRLkEQ\nOpZmPXWttQfwGPrdiFswvPggA5RSq4DjgKVa67+2yyhNSjBvHaC83MIzG4wfMNdPq+PvRcdxVfmT\nAPTvVcOJJ/pIUr5OG6sgCN0Di9/vj2tHpdR84IDWekngeTLwkdZ6VOB5BnAF8BegN/Ah4NBafxvr\nnB6P12+3J27K38qVMC0QiJo2DXJzYft2SHtlJWceeIV81jGEHQDUJaWz84YFDFtya+cNWBCEroIl\n1gttSWmciCHcAGitDwPPBJ4eUEp9BOQAMUX90KHqVl88KyuDsrLDrT6+PcjPh7Iy4/Hkyfb6hUkn\nTuGVeTRYmJTsruK4J+7myIBaam6c1eA8ZrClvRBbzInYYk5aa0tWVkbM19oS5B0HfBZ8opSapJR6\nKPA4HRgNbGnD+ROKaAuTrh6/lfXH1cfX03xV+OcvRD/p7IwhCoLQDYgn+2Us8CAwBHArpa4ALgMG\nAlvDdn0XuFYp9T5gA+7XWu9u9xEnAMGFSclFw9m61s+BoV76Of8JGCtORy6byxcs4sCICQAUFnbm\naAVB6ErEM1H6MZAf5aWbI/bzANe1y6gSnKDXXlcwlZNYzQfLLKSfNii04vTQXhfPP5/Ep72NBUki\n6oIgtBeSY9fB1BVMxXbaKYxRh/numMGA0TFpYcmljCteQnGxnfx8pEaMIAjtgtR+OQqMuq+AuiIv\nSz8+l7u++ylgrDhdwN0A3PD4rWRlyaIkQRDajnjqRwGn04bj4Rm4t+5iNQWh7UFh33DdI504OkEQ\nuhIi6kcBh8PLE0/UUkIun3F6I2G/7PP50gpPEIR2QUT9KFFUZGf3+Km4h+ewLWM0b/epT3Xs4aog\n/e47RNgFQWgzIupHiZwcHzNnurn1vYvJOEvhyhnFi/3re5yWbqmTHqeCILQZmSg9SoQvTir86xSS\nizzUrNzG9n+fwBB2cKJ7C0Puv5xqftNoxakgCEK8iGvYSdQVTOW5ykt4tM89oW3W6mp6LFhA6pNL\nAaSBtSAILUY89U6k18xCbnnvt6xeUcBUigCwu6rosWAB27ZZWbz5l4E96wBphycIQvOIqHciBQUe\n/rJsJJbeyayuoIGwD11+H+NI5g/cyrXX2sjO9vHWW60vgCYIQvdAwi+dTK+ZhVx+T/RUxwXczS08\nQmWlhZISG5MmpUk4RhCEJhFR72QKCjwsq5xG7/GKbRmjGwn7vZb5TGQdAElJEoIRBKFpRNRNQG4u\n/GjtJeRcPryRx97bX8Fj1luZyDo2bLBRWJgq3rogCDERUTcBwe5JN77+Q3b0PIWDg05rkMN+0vH1\nsfQHHnCJty4IQkxE1E3AunVQWJjKrl1WVhy5is+9uQzOrMI7KBuAtNKv+Lf9EtbkL6aoSOa2BUGI\njSiECcjPh4ULXUyYYNyOaS9cwnDtoqoqj1633ARAiqeKKR/O59hj/cDPO2+wgiCYGvHUTUJRkZ05\nc1zMmeOiqMhu1GEv3dmgRoy1uppxa+9j751PdOJIBUEwM+Kpm4RgCzwwBN7ptPFR0WnYD6VwEEuD\nHPaTVtyHb6hPygkIgtAIEXWTEF4bJvg48+kpPDrByHQZNcLLSRtfAYwG1tXzFrB3m5UBCyUUIwhC\nPXGJulJqBLAWeFhrvUQptQIYC5QHdlmstX5FKTUDuBXwAU9prZd1wJi7DUVFdobMKeAk7eabcg87\nelk5t/IlwBD2U/56H7XisQuCEEazoq6USgceA96IeOnXWuuXI/a7BzgTo1jJeqXUaq31wXYcb7ci\nGJJxOgt5+SYbfSrtVESEYtLu/y2ACLsgCEB8E6Uu4CJgTzP7fQ9Yr7Wu0FrXAE7A0cbxdWuCYRiH\nw8sVf78k1DnpJWvDydPwyo4QvbqjVHwUhO5Bs5661toDeJRSkS/NVkr9EtgPzAYGAGVhr+8HBjZ1\n7j590rDbWy80WVkZrT7WbDRny9KlkH7tNPa/Ah8fAE+Ex97z9wvp2bMHjB7NI4/kA1BYWH/8I4E2\nqOHbOorudF8SCbHFnLS3La2dKH0WKNdab1BK3QnMB4oj9rE0d5JDh1pfdTArK4OyssOtPt5MxGNL\ndradWbM8bJ55EY9OqN8eFHYqKnDfNZ993kxwLeNt8nE4PFx4oYdXX7VTXGzcaofDw+2313XYqtTu\ndl8SBbHFnLTWlqa+CFol6lrr8Ph6EfAEsArDWw9yPPDf1pxfaEx4uqNrSiFZhz18tg72cDyzMPLW\nk6orGEQFL3MJd7OAKx74KUr5mDjRG1rY9MADLpTydZYZgiB0MK1afKSUelEpdWLgaT6wEfgAGKeU\nOkYp1RMjni6dlNuZnBwfy5bVsv2My0gdqzh5fF9ey7uLutNOD+3TkyoesN/NwXseBxovbBIEoesS\nT/bLWOBBYAjgVkpdgZEN83elVDVwBPiJ1romEIr5N+AH7tVaV3TYyLspQY89J8dHwdxLSC5ys4pp\nuDcb4ZTkzz4FjLICk533ou/0Q99bmDvX6J4ULuqRE6dSKEwQEh+L3+/vtIuXlR1u9cUlrtaQ5KLV\n2Dd/ScXbX3Dc+ldC26ut6TyadS85f/x5I9EuLExt8HzNmpo2jQHkvpgVscWctCGmHnPOUmq/JDjB\nVMW6gqmUkMua7WMa1ItJ81Uxe988/jv9yZBn7nQaddmLi+0N/kmtdkFIfCTAmuAsXpxMRYWF6dPd\nLH5qBj+oXMV+rNAHJh4yVp/2pIq7Xb9h2W0+eNjw2DMz66tCBpFJVEFIfETUExSn08bixcmhVMX5\n8614vRZWYXTc+N7JHtbvgnF76oV99t57qNnopsYxKzR56nTasFggL89LUZGd22+v6zSbBEFoOyLq\nCYrD4WXnTndI1L3e+hBbxk8upajIx8l1VjiuXthttdWhsgI5ObdQUOAJTZyGPxYEIXGRv+IEprTU\nynXX1bFiRTIA+flujj3Wz+uv2yktv4orWElNjYXjx/lCk6fBsgJXz/gGV2YhBQXnhM5n1JkxYuqR\nk6qxtguCYC5kojSBycnx0a+fnzlzXIwd62HcOB9Llrj4/veNtMdVTGODZwSvfjuGPeMuDh1nd1XR\n40/LybhtNknOhksJFi9OZvHi5EbXirVdEARzIaKewBQUeMjJ8TF3bh2vvloTmuTs18/PddcZsfFV\nTCPvhpOpPPE0Xg4rBGbxerFt30bv6ZeT+uTSRhkxwUyYWNsFQTAnkqduAtrblqIiO5s3G9/XWltJ\nS/Mz7NN/YN+ymSz2h8oKBPGlpVH969/w6cSbmTAhHYB3363iwAEjTp+Z6W+wvakMGbkv5kRsMScd\nkacuMfUuiLHq1N6gXszwWUYXpVxgc/9zGHj4K3pX7wWMOHva/b/l4Bs25sz5ReiYoEeel+dlzhxX\naLtkyAiCeRFR76JEtsdbtCgZ15RCTsLNestw+pdt4oz9r9J36yeAIewXFt/L97/v5fURv+COO1LY\nssUQ9QMHLDzwgAuHwysZMoJgcuQvtJsQjL3DFBYtSsb7iY1SrAzrc3xokVJyXRX2+3/Leb+Gp5+u\nD8UsW1YbCrmEf1kIgmA+RNS7CUExdjptPPVUMpWVRspjLlYOhjXcCIZiLKu28/iVl/HN4IkSchGE\nBEJEvRsRXIVaWWnMsQRXnwYJF/a8L55mfMV/OPzDJbxYPumoj1UQhNYhKY3dCIfDy8KFrtDz0aO9\n7MmbSla+4jNOZzUFodesvvqUx6u+/UNnDFcQhFYgnno3o6jIzpQpbpTysWWLlYICD5s3X0a/w/8g\nPdlDcc0A8jY8Fdrf4qoNlRaouXFWZw1bEIQ4EVHvZtRPmBoCH0p/DDTc2LrWT12qA9s3X2Pbtw+o\nj7Pbtn2Da0ohbsc5TVxBEITORBYfmQAz2ZJctBoA++YvSXrt36FOSgB+mw1f9mDen/k4B0ZMiFoH\nxky2tBWxxZyILdIkQ2gBdQVTeSvzCkrIxX3+BdRecFHotWBpgTPumcqO255o4iyCIHQWIupCIxYv\nTua24umUkMvyT8aylJ83eL2Hv5aZ2+fxtzOfkjowgmAy4oqpK6VGAGuBh7XWS5RS2cAzQBLgBn6k\ntd6rlHIDzrBDv6+1llqtCUJk443bmM4Y64tk8iXrmMhZfTfT46ARZw823ah7SfPxxsuihmOksbUg\nHH2aFXWlVDrwGPBG2OYFwFNa6xeUUrOAXwJzgQqtdX5HDFQ4Olx1VX3jjepqC0v2GYuUSjNOYX3t\nJibyCmfyMWA03ejxp+UMsb/B88OewvHWmQ3OFVmq1+Foe2NrQRCaJh5P3QVcBNwRtu0moDbwuAwY\n087jEjqBxYuT2bnTGireVV5uYcMGG6uYxrv/rOLbxzz88wULuzk+tFDJ4vUyyPsNi0su4akzF+C/\n5VccOZLEq6/aQ18OQQoLU7n99jrx2AWhA4k7+0UpNR84oLVeErbNBrwJ3Ke1fkMpdQQoAk4AXtRa\nP9TUOT0er99ul5hsZ7NuHcyfD2+/bTyfONF4vnQp5OYa2ywW2LYNxm1fSS4l7Px4Pz8+0nCy1Jua\nTlGf63gl9QoyL89n0aKG1ykpgVNPrb8mQH5+x9gkCF2cmNkvrRb1gKA/C2it9b2BbT8D/gL4gXeA\nG7XWH8U6p6Q0GpjBls2brY1qptfnsRs57YsWJZOUBIXuldi3bCafdZzMFo7n29B5PNjYzhB+yv9R\nkjWRYcN8ocbWFguhGjKFhakArFlj3pCMGe5LeyG2mBOzpTQ+A3wVFHQArfUftdZHtNZVGDH4kW04\nv3AUKSqyM2eOizlzXA2aUYMx4bl8eRJbttgoKbHxSpqR8riUWfwfN/AhY0PnseNlGFv5Jxdxddmj\nlJZayMvzMnduHUr5pJOSIHQwrVpRqpSaAdRpreeFbVPAPGAGYAMcwKr2GKTQ8eTk+Bp45eE4HF4y\nM11MmGBsP/10H5nnFfDss0lgJMOQXXg6A9c8HTomlVoWcDdzx20izVGIm3PIzDR+mC1cWH+uBx5w\nNdlJSRCEltFs+EUpNRZ4EBiCkb64GzgWY6K0MrDbJq31TUqpB4BzAR9QpLX+XVPnlvCLgRltCXrP\nwUnNRYvqM1mKi22B/w1hvrHv3zlv4EYc7nWk7d5K76r6cIzXYoMTBnP44SVcvPgHgBGKCRIekjEb\nZrwvrUVsMSed0s5Oa/0xkB/PhbTWdzS/l5AIBNMRg2mIkZ788OG+kLf9o7WXsG2Rl1KlSN/+Na71\nL3HsDqOjks3vhe3b6DH1cn7ITFZxBUUHzmmyk1LkF4ogCPEjtV9MgJlsiVyAlJfniZqGGO65Wyz1\nQuy8dTVVH36C5UAZaSuWNTgmOIla8+hjDJh+dkzxNsskqpnuS1sRW8yJNJ4WOpzI+HmsmHfQc3c6\nbQ36mY6umsZT57vI7VdC3XgHhz74hv4+IxwTnEStu/UyvBuu4fUPruLT3hNDvwYiv1Akr10QWo54\n6ibAbLZEeuHNxbzD0yHHjYNXXjnMtkVrKSqyY9+ymbMoZjKvNTgm6LX/P57GnXd2SLyjpVZ2Fma7\nL21BbDEn4qkLR4WmMmGisWRJEoMG+di1y8r69UEP+zIuKPDz6ATDgx/Sv5aBh7+id/VeoN5rf5UL\nOdT/Gr7ZeBlOJuB02kIrWuPpjSrxd0FoiHjqJiDRbamfOG3oYQc9/n5v/YPjjvdz2fAvOPjP9WRv\neqPB8V6LjW+TT2DhsKc4/ba8Bl8omZl+Nm60MmKEL6pwd2T8PdHvSzhiiznpCE/dNn/+/LaMqU1U\nV9e1+uLp6SlUV5szFa6lJLotSvlYvjyJvDwv559vZ+NGPw6Hl/JyC9df7+b0Hw1nE7nk9NtPep9k\nKivAXVVHD/cRAKz46eU9xKSyF9j5wT5ctjT6nTEYpXz84hc9WLMmia+/tjJ9uid0TafTxrXXpvLJ\nJ3ZKS604nTays/0MHtx+Tkqi35dwxBZz0lpb0tNT7o31moRfhHYhGLLJykph2TIjDh70uIOP65gK\nwCNFpzG07yZ+dKaTlHVvhvZJpZbL9v8f1Uv+zQYe52fPn09JSX1O/KRJaSxYYKRCOhxekpLqBVwW\nMQmCgTTJENqFSAGPhtNpw/HwDH63ZTr/2jWSv3zt4EDu2ZSnDAjtY/V76blvG44FU/n3yT9nIutC\nr/3xj7U4HN5QqYENGwyfpF8/H/fckxx5OUHoloioC0eNcO96FdPIu+Fkkm77KX9L/ynvpZ3fYF+L\nq5YBa5fzbMpMHj7hQWblvhGatHU4vCxc6ArtO2iQn927o3+UnU6b1JYRuhUSfhGOCsEc9KB3PWiQ\nj1lvTWf/fiunHPwH5Vjpke5iYNVXoaqPNr+XbNc33FI6lwMZQ1i14wlgPGBMol55ZR3FxXY2bDBE\nO1pee+TKWEHo6oioC0eFyEVNzz1Xg9ZWlixJYhXTADh2nGKYq4TDez4gZ8froWMtPh9ZFd9w7arL\n8I28m23brGRsO50f/sLB7NnuUNZNeFxdFjIJ3RVJaTQB3cWW8EVNpaUWSkutIdEdNMjH4MFe/nn9\nc9g3f0ll0XtkuXZh37G9wTkO23pj83n42jKMtwZfQ/KZI/lm8ESg8UKpti5k6i73JdEQW2TxkWAS\nGhcFczfy3OsKjAyZtJxT+Po1zXHef2D31pL27XYAMrwVAIzyf0bu9i/YuXMIR667kewpI3ixfFLo\nWk6njeefT2rRQiZB6AqIp24CuqstzZUj+MPZrwAwIXMj/cu/JMVV2SAsE8RvteIbfAKHH16C23EO\nYIRbDhyw8N571UD9QiaIf/Vpd70vZkdsEU9dMCmxyhGE4uFbpgNweODfmfT4/3C+87fs/OMujvjS\nObX649D+Fp8P2/Zt9J5+OdvO/TEP7byS4pLvA/Wx9IICT2j1qUyaCl0ZEXWh04jMbQ+v47JzpzsU\nb5/2wiUo5cVTfgpf/WgEfXZvwqOPUHWwjt4HtoXOYXHVMvQ/y1jc/zXs/IINjOby6d8DYNKktNBC\nJpk0FboyEn4xAWKLQXgdlzGyZh1tAAAe8ElEQVRjjAnO6dPdjUIzyUWrAbBv/pKUotVYKr7Dtm9f\ng3P5sLLDOpQ/ZdzErStymHTvD0LplPFOmsp9MSdii4RfBJMTmX44bFhPKiuNz2xxsa1B+zsgNJkK\n4Mk5BfvmL7F/9GGDkgNWfAz1beU3Fbez/bKhTPTPpjej2TpoAkuXJnHVVcavBPHWha5GXKKulBoB\nrAUe1lovUUplA89iNJj+FrhGa+0KNKS+FaNH6VNa62UxTyoIASJz2JcsqeHHP04Dmq7pEi7uAJ9/\n6KFv9S5O4pvQNhs+TvJv5UF+xTaGsmfcDZT4RrN48bmBazcdX5fSvkKiEU/j6XTgZeAr4POAqD8D\n/FNrvVIp9b9AKfBn4BPgTKAOWA9M0FofjHVuCb8YiC2NG1sHvfPI0Es0kXU6bbxz80uU7rKSSwnT\neIEUXAwLE/cgXqxstwzlMf9sNjC6QYOOaLY4HIZH39mt9dqKfMbMSWeFX1zARUB4U+l84GeBxy8B\ncwANrNdaVwAopZyAI/C6IDRJeCbMPfckM3euIeSRTTqiLft3OLzMSbmCrRiCX8K9nNFjIzOSXmBA\n39oGC5giPXfr2J/Sl5G4OSe0j9NpY+NGK6+/TigkNGlSGtOnu7nxRnf7Gy8I7Uizoq619gAepVT4\n5nStdbCi0n5gIDAAKAvbJ7hdEJolPBPmvvvqGm2PtewfDKHfutUQ9GDJgfv+pxa952769NwIRavB\n5Wok7sPYim/J/+B/6QRqZt6AZ8Qo3I5zQl8cTz0FI0YY+yclwauv2kXUBdPTHhOlsX4GxPx5EKRP\nnzTs9tZX0MvKymj1sWZDbGmawkI4+eR6kf2//7Nz6qnGxzd8+003wf798ILvGuY9AaxcCWeOgZIS\n9i99AZvHReZ39WEZq98H27fRc95d1AwYyl9SZ5O0dTRvk8/FF8MFF8DmzYSKhk2blsH8+ZCfH32c\n69YZ/8d6vTORz5g5aW9bWivqR5RSqVrrGuB4YE/g34CwfY4H/tvUSQ4dqm7l5SWuZlY60pYVK5KZ\nMyf4uD7WHtyutZX0dB+PP15HUZGdsjIP5E8GILmyhi+uuIuzem7EU7S6UY47Ph+pe7Yyy/IrJjOU\nJczGcfFo8uePbVBDZsECIx2yLPw3aRh33VWflmmmSVb5jJmTNsTUY77WWlF/Hbgc+Evg/38BHwBP\nK6WOATwY8fRbW3l+QWhErBWo0bZHNuqoK5jKqALwFHkpIZdepV9ycO1qbB4Xgz31nrvFb4RlHrL8\niu+eG0rywP/Hxi/GMGfOxND5g18m4aIdLTxUUWGhd28/Doe5BF7o2sST/TIWeBAYAriB3cAMYAXQ\nA9gB/ERr7VZKXQHcDviBx7TWf23q3JL9YiC2HF0mTUpj7DcvUl1jIZcSruIF+veppe+hbY329Vut\nVPY9AdstNwDw7BdjGPjDs3E4vI2aXod79KNHe0Mhm7w8T0jg2zOLpiVfFIlwX+JFbGk6+0VWlJoA\nsaVjCe98FO5NA1zBSgDGJG9k1rEvYHW76LkvurjTI5WvOIl/ZPyYXf1GszRQXyY318v06W4qKur/\nzsrLLTzzjDHhGinw7VWiIPJLpSnMeF9ai9giK0qFbk4wm2XNmpoGi5wcDg+rnEa2DHUwn1M57sgm\nLrasJMnfMM/d4vNBdRXD+Zzbq+ey++AQ7NzMBkZTkXQOr75q5/rr3aGwz8yZPUJlf8vLLSFRb48G\n2dIARGgK8dRNgNjSMUSKX16eh+xsP9nZPrS2Ul5uYfhwHytWGKKfnu7nwqpVAKFFTEMH1oRquUcS\nrC/zB99sADy5IzlvwXgcDi9FRfYGAh8U8tJSC9One9oswOGhnkcfrWH69IZzCJGhGTPdl7Yitkj4\nxfSILR1HZPcjra0NFjn17Ak7d1r417+SQvVmABaMfo6Th9nps+czvrfzRbzfHqDKm8SgQP/UcLxY\nqSEVThoK180I5bsHCRf4s89Oo1+/tsfWgytwn38+CYsFPv64qsHrkaEZs92XtiC2NC3qtvnz57dl\nTG2iurqu1RdPT0+hurprdLIRWzqO5cuTyMvzkpfnZcsWK9dfX794aNIkL+XlFm67zc3LL9vYu9cK\nQH6+m8xzchg9YxR7PtuL+pmDb8dM5uN1NYCf2qRe9PYdCp3Hip9k3CQf2k/S22+R/N9i8PlI+ng9\nFpeLk8/Pxum08Ytf9GDDBjulpVacThvZ2X4GD27ar3E6bZSWWhvt53TaKC62UVJio7LSEjpfaamV\nX/yiB8XFDa+Tk5NkqvvSFsz2GWsLrbUlPT3l3livSUxd6NLESoMMkpnpp7AwNVSWd9AgHwMG+FHK\nx/z54HZfSUZmHc+/lsT3xtt5U1/BgINfMo0X6J3sYlBdw/oywYYd6fPugh6peIYOxTV9BvkjRpG5\ncGIonh9vbD04H2CUU6oPp8ydWxf4FdL4fOHzBu0RwxcSCwm/mACxpXOJDNGsXWunuNgWisX36uWn\nVy8fn3xSzebNVh6d8E8AHvzJpxzz2uomQzNQ327v5SE/B2DPsaex7YSJTfZMjZwP6NXLmAt46636\nBXux2gFG256I9yUWYotkvwhCkxQV2Rs0qI70gisrLVRW2igsTCU728+QOQUAvL3FTcH8HPbutvDF\nvCJyKYk6sRr03qfsuBNLaiqVx57Iuv3XkPqkt1H8PUhkOeLKSgslJbYGmS4tWYwFUka4uyCeugkQ\nWzqX8InM4ONFi5JJT09hx466UHZM+ESr02nD6bThcHh5/vkkLq428t0HlG9i0oFVWA8cwJ+UhG3f\n3qjX9GLFmmYI/M5zf0T2lBEhcQ+Kb/D/AwcsDcbQmnBKU2WEE03sE/EzFgvJfglDbqw56Sq2FBXZ\nmTkzldtvd6G1FaV8DUIcwewSMET3vfeMsMjn9xRxxhmGOB5Ztoa6TzfjrY1e2z2IFysMOYHDDy8B\n4E+3fclX6aM5/ba80BeM1lZSU/1YLPDYY66Y54qG02njkUfSePtt43lwAVSQ8Dz+RKCrfMZARL0B\ncmPNSVezZdmymgZefGamv9GqVIi+UjS5aDV7dlv4+7ytTOMFTu69H789iZTy6N57XVI6O5JOIqt6\nB9sZwr/6X8OYMT7cuaM4c24ehYWpDb5AWsL+/RmhSpZBb3/SpDRKS62hVM72XO3akXS1z5iIegC5\nseakO9gSPrEaJDwsEh7OWLQomVH6RWMnCwz75EUyv/2SY7xl1BF7chUMD95lTcU9+ERWcC3bttua\n7dYUi6VLM6iqMjz80lILpaXWRl9MrQ3tHG26w2csjuNkolQQ2ovgxKrTacNigbw8b4PqjeHdmXJy\nfEyeOyV0XPVxfg7VwjMrkpnGC+RSQrrVxRBf4/CMDR9pvirY/gU3W+ZSTSpfM4xeY3/IsRv9eGg4\nyRorNh7cHt5Navhwd2gStn9/H9dc425gg5C4iKgLQgsJZpeEl/ktKrI32Z0puB8FBSxalEzP6yys\nXAErgQev/ZRvPt/M4MObqDlQQ4/sTJI++7TBNa1+Hz2pYjSf4XvsC3wpqfiHGTnwAJ4Ro1i8+AdA\n42baixcnk5QEs2bVj2P27BQGDfKxa5eVffusDfrCtpVEm3jtakj4xQSILeakNbZE5rxHC2cUFdnZ\nvNlYvaq1lUnlqxh6+6VMKl/F/rXreefTPpy1axVZlOG1JDHQHztE47da8Sal8o3tJB6v/gkAA/p7\nOf7ikWRMObtR7Ztg2Mbw1n3NjrU1tKR6ZGvo7p+xwHESUzczYos5aY0tsRYEQUMPtql6MAeeWMM9\n83oA8Hj+37DrzfR2lVHrTyLtUPRJVqivQeO32vm2xwkc9+sfsm+fhZlLvsfb5Dcq/BU51qCn3loP\nO1oBtfaceA2+f4WFad36MxY4TkTdzIgt5qQ1tkTmvGdmGh/xaE01YolgeP33UVteRA33UVpqoc9r\nq+l/cBNZND/JCobIu22plPc5kWXe6/H5fNy6IicUh48c6/LlSaHxhY8hOP54wirx/FJpLcH3z+m0\nd+vPWOA4EXUzI7aYk/awJbytXTQPNpoIxhLbpy94LuTBzz3heQYc3ETP6qZLFAQJevF70k7CNX0G\nQ4f6QqtZo325BMccZM2amrjCKk39UmktkeObOBFuvbW6XX4BdHb8X0Q9DBEPcyK2GEQK0ejRnlDR\nsHAPtqkQSOQ5Bg3yMXiwj7w8L6O2vMhxx/k5bqAvVKJgoK2MFEsdtfY0Mmtjh2mCcfiqgSdim3k1\nQIMwTXinpiBpaX6qq5vPZ2/ql0pbCP/yKymBrKz2+Yx1dPy/OUTUwxDxMCdiSz3hQvSTn9SFBC7c\ngw2KYLDsQHGxIabR+p7ed18tP/uZu8Fxs2en0Of11ZQfNCZez7YXk5NayvcHbsS95wDJ6bFLFUB9\nm75aj52DvQbz0YhrqKqy8OF6OxsYzdvkA6CUF62NsbUkrBIUzaC9rRX38C+/nj1TuOmmtn3G2jP+\n3xZv3zSirpSaCVwTtukM4CMgHQhW6/+V1vrjps4jom4gtpiTtnrqzz+fRHa2IX5btlh5+ulaoKE3\nGyTW6s7w2Ha0cEZkFgsYfVfVcC+nn+7lrJ0vkrb9S/q4D+C12ZsUeAC/xUqNJRWL3U6p/QT+kfFj\namosVFRa2cBotg6awNlne3j00aZLFcRTZbIlLFqUjMPhxeHwsnRpBqNHtz380l7x/7Z4+6YR9XCU\nUhOBK4FcYLbWemO8x4qoG4gt5qSltoR7bJFL+qMJefCYaGUHYsXXo50j6MVGFv66444UJh1Yxe1z\n6+jdK5XaZ/+GXX+Jb98B9lckNxuHB/BZrPhTUjlca2c7Q+h1s5FRM3Kkr0GFyWjearSVt015xE15\nvOHCOW1aBm63p926R0Hr4v/t4e2bVdTfAGYAzyOi3irEFnPSUluamxSNRbQwTXMiEy09Mrzw1/vv\nG52PgmP43e/snL71z4DRDeqsnasYWr2JntVlpCW5qbWnNZkuGcRnsWJJTcVvt+HNHhxa/LRsWRJf\npY9m3ltnhvaN9WUTyyOO5vE++WQSzz+fREmJYW+vXv5W1aqJ9oURzxdmc7TV2zedqCulxgGztNbX\nKaXWAQeBfsCXwK1a6ya/Sj0er99utzW1iyCYnnXrYP58QlUQzzgDPvrIeFxSAqee2vTxwY6SJSUw\nYgTMmwcrV8K0abGPyc+vv3aQ8GMeegh+9avoY8jNNUI0986H99+H8bwPpaXUflJC5TdlJOHGbU/j\nWE/znrzfYqXWkorLZ3jy7510HZMnw7BhsO670eTPz2f+fGMMubmGRzxvXv3YN2wwHq9ZU//+TZxo\nvCf5+ca/qqr693PtWrj00uh2NUW09yv4OPhac0TbP7wbaLhtR4EOE/Ungee01uuUUlOBz7XWW5VS\nTwBbtda/b+p48dQNxBZz0hJb4pkUjUVLPMZ4f/JHhhYWLUphzZrqmMcmF61m+bIktLZy8sEPyKaU\nkZYS+lvKOCa9Dn9aWrPxeIjuyX/xhTUUrnmxfFLovVm8OJmNG22MGOFl4UJXA4/3wAFLo8wfh8NL\naamF88+3U1Xliuu9ber9amksPNr+bfX2zeipa2Ck1rouYvtFwFVa62ubOl5E3UBsMSctsSVcRJub\nFG0r8ZYiCM+qWbw4hbKywzz3nJ1bbkmNemz4pOsVrAxtv2ygkwlDdtK/fBOHt5Zj8bpx2VLp721e\n5IPZNUGRr1rwALfdlkJm6ed4vIYubWA0m7ImcO65XrKz6+vWR7PznnuSeeIJw5Z439vI80R+YUT7\nYgwP13TkSllTibpS6jjgJa31WKWUBXgNuEJr/Z1SahFQprVe3NQ5RNQNxBZz0hJb2iM+Gy8tmeCL\nXIU5ZowhbtOnuxsc21y3peS1q3n++SRKd1k5i/cNT54SBqfthzo31aSR5Wle5Kst6WzxD2MI20Pb\ntjOEfZNnhETy7YrTOXNuXkw7W3JfIrOQmvrCiPa+RUstbc+VsmYT9bHAAq31hYHnVwJ3YKQ07gZm\naq2bzF8SUTcQW8yJWW2J5wsk0rs87TTweLyhCce8PA95ed5QOd6giF1/vbvBpGt4x6dwYQt68mmp\nfs5NfZ+LRm1n/zpNFmX0sNZR6UuPK7smiA8rvpRUrCmNwzYAX3xh5ZQfGi3/srIyWLPGkJbmvOVY\nWUjhXxilpRamT/c06ZU3l1raWkwl6u2BiLqB2GJOEt2WyFWY5eVVjbzNWCJWXm6J2rfV6bRRXQ2f\nfdYwBfPGvn9H5fjol+nn2K0fULFpD7mUcHLv/eB2U1EXnycfJDxsEyQo9j179uAPf/BSUwNnz85F\n3ehodHxzIZOmCqrFU7qhvX6JiaiHkeh/cOGILeYk0W2JXIV55Ej9gqFwbzPe0EKwfvzmzVa0tnL4\nMKxbZ9SlCV/tOnNmD6YFY/IWOMv/Puzczaef2cilhP7pVVitkFz9Ha6kpksaRBKebQOw0zoEy3VX\nM3Ro/ZiD+fPN2dUeXnlba8dI5yNBEOIm2MwDYN26FCorfQ28zSDBTk7Bx7FErN47tTN3bh0339yD\n8eONsMXhw4bGOJ02lGrY7em65VcysvRFMuYU4tIvklHyAZP+ZyyWjz6kYv0eehzYRNWOcpJwk5aZ\nGrOHK4DF7yPVX0Ww7fcxvs/wLv8Cb1Iq9lRDYIMe/cE3bKzJN/bbuOQ01GPjG5zL4fCSmekKdYB6\n4AEXSvka/UppSrjDu1w1x9EqHiaeugkQW8xJd7GltaGF8LBF8LjwCcZ4QjvJRaspWmu8PnjPBxxz\neBe5lGA9cADc7tC14k2pDO1vteKxp2LrYQjpoV4nkHLj1WzbZuXbby0MHOhn6FAfz34xhm8GTwRi\ne+XBRWULFrhCgtyajJjI98bY1rra8BJ+MTliizkRWwwiPcxIQcvN9fK97xllhCNFLjPTH3f6JcDn\n9xRxxhnGdewffRjax7Z7N3b9JYe3lmPzuUlK8lNtbXnopoZU6vx2bDZIT/eHxB4ITcxu22alYugo\nDoyY0Kh+zZ/+VBN6H5pKD418/2KVNm5tbXgRdZMjtpgTscUg2qKb8Hj16NFe0tL8jRYQKeVrVX2V\naGGKbYvWUlRkR2+xcRbv0/cYPz84ZQe1n2wmw2Wsgk3r4cfeu2UePUTk0nss1NTC3kDnqI8/trJ6\nTb0NntyRnLdgPA6HN2Z6aDQ7It+vYGnj1taGF1E3OWKLOenutjQVYli0KJmdOy0UF9vZtcuoMRNc\n9Rm+gKg1oZ1YKz3DhfHRR2vYt2QtektAOK3FpKTAT87bzrevaY5xl2H3GyLrT0uj15GWCT3Ud47y\nWe3UBSJB9pOy2TbxGt5808a27ca1hw7xknzmSK5Y0jBmH152ODxXvrzcwjPPGF8Ura0NL6JucsQW\ncyK2xM6MCQp0U/XeW9ogI1pYJzyOHfT609NTqK52MWWKp1EVyB+nvkB1jaF3Z/E+ACfaSxlhKaG3\n2/DobVY/FsDXI43e1S0Xe7/VitueSnWdMc6e6X78Q4zJ2W3bjBLFH3xgw17yBQA9UvxszRjF45uM\nYmczZ/YIvY+trQ0vom5yxBZzIrY0v3q1qdfjra3SVJjiP/+pX78Y/CLJyspg2bIaNm82fiGEr379\n85+r+fGP0xqcP7hQqmdPHz/IeJ/jjvfTt4+f9IO7GHx4E9YDB6j6zo0f8PosVJHWooVTQfxWI2a/\nuedYjj3WT8+vPw+9tp0hvD3kGs4918uRI4Rq4bzmvoj8fBF1QP7gzIrYYk5aa0tz4ZNor7c0MyRc\n/GfPTmkQ0ol2bNCWaCWHS0psXHihMQaLBYYP9zUq+xs8LrloNXqzlaIiO322GJOyPXr4GdNvZ6gs\ncbLVTXD9U0szcCLxW6x4k1PxYEzU+kePJNn5rkyUBpE/OHMitpiTo21LPAuaool/Xp6XggJPk8dG\n2hKeTrl5s4Vly1yUl1tC44gsdxC8NjT+dfDuu1WM1C/y0Uc23nrTRu7h/1JwqfGl9d1ne6j7dDPp\ntUYYB2j15GyI007ju/n/G2o2Ei8i6iZHbDEnYkvriTfrJZr4N3dspC3RWucFUw9j/dII/3UQvN7O\nnRYsFpg+3RP1l8ak8lXs2W3hnnk9OIv3GXGqF7cH0g7spv/BTWRRRjXp9Ojhp6bWggUYkHKIpGNi\ni/6Hz5Qw9OLs+N7Uhu+BrCgVBOHoEb6aNXz1aiTRVrPGe2wQh8PLzp3ukAhXVloCKz3rGoSLooWG\nCgtTQ4XNgsW/HnvMFXWlaR1TWb4oGdcUK3tVAfssoJSPge/9g/krrgpd53x7MRkD/XjOOJPB337A\n9wbuxB9oI1hVEfD6OYY+o7M5c8dKyvhlvG9rXIinbgLEFnMitnQ8TcXsw2uwQH0WTTRbFi1Kjrtt\nXjz11bOz/Y3K9cYab+S1I7OAghO1RWvtoQVVJRlnoXJ8XPOjJMryJ7fsTUPCL6ZHbDEnYkvnEgyR\nBAlm0USzpajIHsqG0dpKTo4vZsgnWngnUui1tsadX9/UtSN7yYLxiyH4eObMVCnoJQhC1yYyRBKk\nsDCV22+vo7Cw8TGG6NaLb7BMcLSMm2jhnaaKmjW3YCry2uEho/CCX5GhoI5CRF0QBFMRWT0xSDC2\nHYtwoSwutlFcbItaPTGauMaK48dbWbG52P2kSWlMn+7mxhvdsU7Rblg7/AqCIAgtJOg5jx/vIS/P\nw5w5rrgmTZ1OG4WFqRQX2ykutlNYmNooNh+NWF704sXJIW+7JTgcRjPtIElJ8OqrR8eHFk9dEATT\nEBTgoOccLQ7dFLFqpLdmHJFZMuEhmXhKHxQV2bnyyjqKi+2hAl7B83RkTXWZKDUBYos5EVuOPvGU\nFmjOltZUhoxGtBz6eEsfQPT6OJFZOabpfKSUygdWAiWBTV8Ai4BnARvwLXCN1toV9QSCIAhhxPKM\nW+PRtjTPPRbhk6dLliRRWmpt0fjCxxBPZ6n2oi3hl7e11lcEnyilngGWaq1XKqX+F7geeKKtAxQE\noevTXmETiB0fbymRXw7Dh7tbNb72+pKJl/acKM0HigKPXwLOa8dzC4LQxQl6tPFOinY0kV8OrR3f\n0UplDNKWd+5UpVQR0Be4F0gPC7fsBwY2d4I+fdKw25ufmY5FVlZGq481G2KLORFbjh5nngnTphmP\nV66ErKyUmPt2hi0tGV9LaG9bWjVRqpQ6HjgbeAE4EXgL6Km17ht4fRjwZ611XlPnkYlSA7HFnIgt\n5kRs6YCJUq31buDvgadblVJ7gXFKqVStdQ1wPLCnNecWBEEQWk+rYupKqRlKqTmBxwOA/sAzwOWB\nXS4H/tUuIxQEQRDiprUx9SLgb0qpS4Fk4OfAp8CflVI3AjuAP7XPEAVBEIR4aW345TAwJcpL57dt\nOIIgCEJbkNovgiAIXQgRdUEQhC5Ep9Z+EQRBENoX8dQFQRC6ECLqgiAIXQgRdUEQhC6EiLogCEIX\nQkRdEAShCyGiLgiC0IUQURcEQehCdH4l+laglHoYOAvwA7dordd38pDipiu0AlRKjQDWAg9rrZco\npbKJMn6l1AzgVsAHPKW1XtZpg45BFFtWAGOB8sAui7XWrySILYuAczD+ru8H1pO49yXSlgIS8L4o\npdKAFRhFD3sAvwU+owPvS8J56kqpicDJWuvxwEzg0U4eUmt4W2udH/h3M3AfRivAc4CvMVoBmhKl\nVDrwGPBG2OZG4w/sdw9GB6x84DalVN+jPNwmiWELwK/D7s8rCWLLJGBE4O9iMvAIiXtfotkCCXhf\nMGpkfaS1nghcCTxEB9+XhBN14PvAGgCt9ZdAH6VUr84dUpvJJ3FaAbqAi2hYLz+fxuP/HrBea10R\nqLHvBBxHcZzxEM2WaCSCLe8Agb48fAekk7j3JZot0Vqkmd4WrfXftdaLAk+zgV108H1JxPDLAODj\nsOdlgW2VnTOcVtHmVoCdhdbaA3iUUuGbo41/AMa9IWK7aYhhC8BspdQvMcY8m8SwxQtUBZ7OBP4J\nXJCg9yWaLV4S8L4EUUoVA4OAS4DXO/K+JKKnHknMtk4m5SsMIb8UuBZYRsMv10SzJ5JY408Uu54F\n7tRanwtsAOZH2ce0tgR6HMzEEL1wEu6+RNiS0Pcl0NqzAPgLDcfZ7vclEUV9D8a3WpDjMCYbEgKt\n9e7ATzK/1norsBcjhJQa2CURWwEeiTL+yPuUEHZprd/QWm8IPC0CRpIgtiilLgDuAi7UWleQwPcl\n0pZEvS9KqbGBRAIC47cDhzvyviSiqP8HuAJAKTUG2BNo2pEQdNFWgK/TePwfYPStPUYp1RMjPvhu\nJ40vbpRSLyqlTgw8zQc2kgC2KKV6A4uBS7TWBwObE/K+RLMlUe8LMAH4FYBSqj/Qkw6+LwlZelcp\ntRDjzfIBs7TWn3XykOJGKZUB/A04BqMV4L0EWgFipDztAH6itXZ32iCbQCk1FngQGAK4gd3ADIy0\nrQbjV0pdAdyOkXr6mNb6r50x5ljEsOUx4E6gGjiCYcv+BLDlBoyQxJawzdcCT5N49yWaLc9ghGES\n7b6kYoRYs4FUjL/3j4jy995etiSkqAuCIAjRScTwiyAIghADEXVBEIQuhIi6IAhCF0JEXRAEoQsh\noi4IgtCFEFEXBEHoQoioC4IgdCH+Pwr3nGJwee5kAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "X9uiFJ-ogBRi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Loading a Model"
      ]
    },
    {
      "metadata": {
        "id": "Xo1mKz1aCmWf",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "unlabeled_images_test = pd.read_csv('gdrive/My Drive/dataML/test.csv')\n",
        "#unlabeled_images_test = pd.read_csv('test.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "SB5PoV4uDu_0",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "X_unlabeled = unlabeled_images_test.values.reshape(unlabeled_images_test.shape[0],28,28,1)/255"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "dbtjpi8JgBRj",
        "colab_type": "code",
        "outputId": "38d35ea3-7c67-4544-80f8-6d93388ddf22",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "with tf.Session() as sess:\n",
        "    \n",
        "    # Restore the model\n",
        "    saver.restore(sess, 'models_saving/my_model.ckpt')\n",
        "    \n",
        "\n",
        "    # Fetch Back Results\n",
        "    label = sess.run(Y, feed_dict={X:X_unlabeled})"
      ],
      "execution_count": 265,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Restoring parameters from models_saving/my_model.ckpt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "9KWKoSzEgBRq",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "label = np.argmax(label, axis=1 )"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "hCR-AZTmgBRt",
        "colab_type": "code",
        "outputId": "15cd3ce6-3b5d-4d1e-ff47-c470d7594ecf",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "label.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(28000,)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 147
        }
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "u13ffZdjrqQf"
      },
      "cell_type": "markdown",
      "source": [
        "### Predict"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "In9n3ZirrqQg",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "unlabeled_images_test = pd.read_csv('gdrive/My Drive/dataML/test.csv')\n",
        "#unlabeled_images_test = pd.read_csv('test.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "ErPyp1xIrqQi",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "X_unlabeled = unlabeled_images_test.values.reshape(unlabeled_images_test.shape[0],28,28,1)/255"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "be1e8ef0-ff29-43e5-bc02-b33c5b515d43",
        "id": "0f6xn42MrqQk",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        }
      },
      "cell_type": "code",
      "source": [
        "plt.imshow(X_unlabeled[1].reshape(28,28))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7f077569d588>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAD4CAYAAADFJPs2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADshJREFUeJzt3XGMVOW5x/HvAoHKykKx2e6t1IC9\n5Ym48gcbA9tcWnqh19a0+Ac0highqwlXLdikqQkNCVGDltQYbi4qCXJZbzAkoiaIrZqit5bEYLrB\nbqUreWyvlQRXRK0tUpTLsnv/2GHd2d05MztzzszI8/v845z3nXPmybg/zpnznnPehoGBAUTk4jah\n1gWISPYUdJEAFHSRABR0kQAUdJEAJlXpc3RqXyR7DYU6yg66mW0FFjEY4h+7e1e52xKRbJV16G5m\n3wK+7u7twK3Af6ZalYikqtzf6EuBfQDufhT4opk1pVaViKSq3KC3AO8PW34/1yYidSits+4FTwKI\nSO2VG/Re8vfgXwHerbwcEclCuUH/NbASwMwWAL3u/nFqVYlIqhrKvXvNzLYA3wT6gR+5+x8S3q5x\ndJHsFfwJXXbQx0lBF8lewaDrEliRABR0kQAUdJEAFHSRABR0kQAUdJEAFHSRABR0kQAUdJEAFHSR\nABR0kQAUdJEAFHSRABR0kQAUdJEAFHSRABR0kQAUdJEAFHSRABR0kQAUdJEAFHSRABR0kQAUdJEA\nFHSRABR0kQAUdJEAFHSRABR0kQAUdJEAJpWzkpktAZ4EenJNR9x9fVpFiUi6ygp6zm/dfWVqlYhI\nZnToLhJAJXv0eWa2H5gJ3OPuB1KqSURS1jAwMDDulczscuBfgL3AlcBvgH929/8rsMr4P0RExquh\nYEc5QR/JzH4H3OjufynwFgVdJHsFg17Wb3Qzu8nMfpp73QJ8GXinvNpEJGvlHrpPA/YAM4DJDP5G\nfy5hFe3RM3Dq1KmCfT09PQX7stDe3s6hQ4eq+pmFXHrppXnL11xzDUeOHBl6fREruEcv62Scu38M\n/KDsckSkqjS8JhKAgi4SgIIuEoCCLhKAgi4SQCoXzJRAw2tj2LdvX2J/b29v3vIdd9zBI488MrSc\nNIT26KOPVlZcEf39/XnLfX19TJr02SDOhAm124c0NzfnLR8/fpxZs2YBsHHjxsR1ly5dmtg/d+7c\nyorLVroXzIjI54uCLhKAgi4SgIIuEoCCLhKAgi4SgIIuEoDG0StQ7LbMxx9/PLF/7969if0fffRR\n3nI9jVXX8zh6Um3F6mpra0vs37NnT2L/nDlzSqgwMxpHF4lMQRcJQEEXCUBBFwlAQRcJQEEXCUBB\nFwmgkimZwuvu7k7sz/qecEnf4cOHE/tPnjyZ2F/jcfSCtEcXCUBBFwlAQRcJQEEXCUBBFwlAQRcJ\nQEEXCSD8OHpfX1/e8qRJk/LannrqqYLrrl+/PrO6ACZPnpzYNn/+/Ew/P8lVV101qm316tVDrzs7\nO6tZTp4TJ06Majt+/DjA0PPdoykp6GbWCjwDbHX3h8zsq8BuYCLwLrDa3c9mV6aIVKLoobuZNQLb\ngJeGNd8LPOzui4E/A7dkU56IpKGU3+hngeuB4fMDLQH2514/CyxLtywRSVPJz4wzs7uBD3KH7ifd\nvTnX/jVgt7t/I2H1i/KZcSJ1puAz49I4GVdw458HlZyMu/nmmzOrC0afjDtz5gxTp04dWq6nk3Gd\nnZ10dHTkLdfKyJNxLS0tQ22Vnox75ZVXEvsXLlxY0fazUu7w2mkzuyT3+nLyD+tFpM6UG/QXgRW5\n1yuAF9IpR0SyUPQ3upm1AQ8Cs4FzwDvATcBjwBeAY0CHu59L2Ezd/kZPOswDuOKKK6pd0pAFCxbk\nLb/66qssWrQob1lGO3XqVN5yU1PTUNvwsf6xPP/884n99913X2L/nXfemdg/ZcqUxP4Klf8b3d0P\nM3iWfaTvVFCQiFSRLoEVCUBBFwlAQRcJQEEXCUBBFwkg/G2qcvFpamoq2LZhw4bEdYsNr23cuDGx\nf82aNYn9zc3Nif1Z0R5dJAAFXSQABV0kAAVdJAAFXSQABV0kAAVdJACNo2fo/vvvT+xfvHhxYn9j\nY+Ootp07d1ZUU3RXX311Yv/atWsT+3fs2JFmOVWjPbpIAAq6SAAKukgACrpIAAq6SAAKukgACrpI\nAOHH0ZcuXZq33NPTM6qtkE2bNiX2F5tWuZxH/7a2to57HfnMWPeqD1dsnL2YZcuSpyF8/fXXK9p+\nubRHFwlAQRcJQEEXCUBBFwlAQRcJQEEXCUBBFwkg/Dj60aNHE9tmzpxZcN358+cnbjvjKXKlDJ9+\n+mli/3vvvZfY39/fn9j/xhtvjLumaigp6GbWCjwDbHX3h8zsMaAN+DD3lgfc/VfZlCgilSoadDNr\nBLYBL43o+pm7/zKTqkQkVaX8Rj8LXA/0ZlyLiGSkYWBgoKQ3mtndwAfDDt1bgMnASWCdu3+QsHpp\nHyIilWgo1FHuybjdwIfu3m1mG4C7gXVlbqumJkzIP6jp7+/Pa0s6Gbdr167EbS9fvryy4iR1xU7G\nbdmyJbF/8+bNFX1+X19fReuXq6ygu/vw3+v7ge3plCMiWShrHN3MnjazK3OLS4A/plaRiKSulLPu\nbcCDwGzgnJmtZPAs/BNmdgY4DXRkWWSWRh66j2xbtWpVwXV1aF6fDh06lLfc3t4+1Hbw4MHEdYs9\ni3+sv5fhkv5eaqlo0N39MIN77ZGeTr0aEcmELoEVCUBBFwlAQRcJQEEXCUBBFwmg5EtgK1S3l8BO\nmpQ/8NDX15fXtnDhwoLrdnZ2Jm577ty5lRUX1MjhsZG6u7sT+0c+Znv4/9Niw2OVOn78eGJ/c3Nz\nlh9f8BJY7dFFAlDQRQJQ0EUCUNBFAlDQRQJQ0EUCUNBFAgj/uOdiurq6CvatWbMmcd09e/Yk9s+Z\nM6esmqqh2JNWjh07lre8fft2br/99qHlSsarDxw4kNj/9ttvl73tShW7jXX69OlVqmR8tEcXCUBB\nFwlAQRcJQEEXCUBBFwlAQRcJQEEXCUD3oxe5H72S8eDZs2cn9ifNAgNw/vz5vOWuri6uvfbaoeWJ\nEyeWXVsxPT09if2ffPJJ3nKa31ulRk5tPJ770Tdt2pTYf9dddyX213iqbN2PLhKZgi4SgIIuEoCC\nLhKAgi4SgIIuEoCCLhJA+PvRx3oO9/C2WbNmlb3tt956q6L+kePBAIcPHx56Xcux6pFj/IXasjBv\n3rzE/ra2tlFtq1evBoo/i/9iVVLQzewXwOLc+38OdAG7gYnAu8Bqdz+bVZEiUpmiuwQz+zbQ6u7t\nwHeB/wDuBR5298XAn4FbMq1SRCpSyrHfQeCHudd/AxqBJcD+XNuzwLLUKxOR1IzrWnczW8vgIfx1\n7t6ca/sasNvdv5Gwat1e6y5yESl4rXvJJ+PM7AbgVuDfgD+VsvHPgxMnTuQtt7S05LVVcjKuUiNP\nxvX39+edgKunk3EDAwM0NHz2p5DlDTfjPRnX2dlJR0fH0OuISvpLMbPrgI3A99z978BpM7sk1305\n0JtRfSKSgqJ7dDObDjwALHP3v+aaXwRWAI/n/vtCZhVmbOrUqYltwx9hPNJrr72WuO2kR0WXq172\n6DNmzBjVdtlllw29XrVqVWafXeyRy9OmTRvVFnVPfkEph+43Al8C9prZhbY1wE4z+3fgGPDf2ZQn\nImkoGnR33wHsGKPrO+mXIyJZ0CWwIgEo6CIBKOgiASjoIgEo6CIBhL9NtampKbFt27ZtBdd98803\nE7f98ssvl10XjL5qD/IfR7x58+ayt33bbbcl9re2tib2t7S0jGrbtWvX0Ovly5eXV5hkQnt0kQAU\ndJEAFHSRABR0kQAUdJEAFHSRABR0kQDCT5tcz86ezX+w7pQpU/Lauru7y952sae0jHVPt9Q9TZss\nEpmCLhKAgi4SgIIuEoCCLhKAgi4SgIIuEoDG0UUuHhpHF4lMQRcJQEEXCUBBFwlAQRcJQEEXCUBB\nFwmgpOe6m9kvgMW59/8cWA60AR/m3vKAu/8qkwpFpGJFg25m3wZa3b3dzC4Dfg/8D/Azd/9l1gWK\nSOVK2aMfBH6Xe/03oBGYmFlFIpK6cV0Ca2ZrGTyEPw+0AJOBk8A6d/8gYVVdAiuSvcovgTWzG4Bb\ngXXAbmCDu/8r0A3cXWGBIpKhUk/GXQdsBL7r7n8HXhrWvR/YnkFtIpKSont0M5sOPAB8393/mmt7\n2syuzL1lCfDHzCoUkYqVske/EfgSsNfMLrR1Ak+Y2RngNNCRTXkikgbdjy5y8dD96CKRKegiASjo\nIgEo6CIBKOgiASjoIgEo6CIBKOgiASjoIgEo6CIBKOgiASjoIgEo6CIBKOgiAZT0hJkUFLx9TkSy\npz26SAAKukgACrpIAAq6SAAKukgACrpIAAq6SADVGkcfYmZbgUUMPgL6x+7eVe0axmJmS4AngZ5c\n0xF3X1+7isDMWoFngK3u/pCZfZXB6bAmAu8Cq939bJ3U9hh1MpX2GNN8d1EH31stpx+vatDN7FvA\n13NTMF8F7ALaq1lDEb9195W1LgLAzBqBbeRPf3Uv8LC7P2lm9wO3UIPpsArUBnUwlXaBab5fosbf\nW62nH6/2oftSYB+Aux8FvmhmTVWu4fPiLHA90DusbQmDc90BPAssq3JNF4xVW704CPww9/rCNN9L\nqP33NlZdVZt+vNqH7i3A4WHL7+faTlW5jkLmmdl+YCZwj7sfqFUh7t4H9A2bBgugcdgh50ngn6pe\nGAVrA1hnZj+htKm0s6rtPPCP3OKtwHPAdbX+3grUdZ4qfWe1PhlXT9fA/wm4B7gBWAP8l5lNrm1J\nierpu4M6m0p7xDTfw9X0e6vV9OPV3qP3MrgHv+ArDJ4cqTl3fwd4Irf4v2Z2Argc+EvtqhrltJld\n4u6fMFhb3Rw6u3vdTKU9cppvM6uL762W049Xe4/+a2AlgJktAHrd/eMq1zAmM7vJzH6ae90CfBl4\np7ZVjfIisCL3egXwQg1ryVMvU2mPNc03dfC91Xr68WrNpjrEzLYA3wT6gR+5+x+qWkABZjYN2APM\nACYz+Bv9uRrW0wY8CMwGzjH4j85NwGPAF4BjQIe7n6uT2rYBG4ChqbTd/WQNalvL4CHwm8Oa1wA7\nqeH3VqCuTgYP4TP/zqoedBGpvlqfjBORKlDQRQJQ0EUCUNBFAlDQRQJQ0EUCUNBFAvh/e2//S7Ri\njc4AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "e287e92d-d7c5-4e6b-cc37-711face336b9",
        "id": "3QYd9lBwrqQo",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "X_unlabeled.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(28000, 28, 28, 1)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "metadata": {
        "id": "ojRKNc76gBRx",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Predict the unlabeled test sets using the model"
      ]
    },
    {
      "metadata": {
        "id": "yQrGiou8gBRy",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "imageId = np.arange(1,label.shape[0]+1).tolist()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "9Oj02pY3gBR1",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "prediction_pd = pd.DataFrame({'ImageId':imageId, 'Label':label})"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "VLjMgeXEgBR4",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "prediction_pd.to_csv('out_cnn4.csv',sep = ',', index = False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "76xhwmO1gBR6",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}